diff --git a/openspec/ROADMAP.md b/openspec/ROADMAP.md index 7743e05..dd82a2c 100644 --- a/openspec/ROADMAP.md +++ b/openspec/ROADMAP.md @@ -41,6 +41,14 @@ | 29 | `29-beta-label` | 22a | Метка beta и дисклеймер; версия 1.0.0 из package.json на фронтенде; правило semver: каждый change поднимает версию (config.yaml, CLAUDE.md, CHANGELOG.md) | | 30 | `30-rack-front-plate` | 28, 29 | v1.1.0. Передняя панель корпуса: крепление keystone по моделям (гнездо с рампой и защёлкой, обойма за панелью), группы отверстий (колонка, ряд, сетка, зоны вокруг корпуса), «отверстия» вместо «кабельных», окно по умолчанию, ссылки на референсы в README | | 31 | `31-rack-shelf` | 26, 29 | v1.2.0. Полка: передняя панель с ушами лицом на столе, пол с вентиляцией (щели, можно глухой), боковые стенки с прямым скосом, вентиляция передней панели опционально | +| 31a | `31a-status-log` | 29 | v1.3.0. Общий независимый лог приложения: статус-бар показывает лоадер и текущую строку, по клику открывается панель с полным логом (уровни, источники, фильтры, копирование); лог пишут генерация, импорт, хранилище, фото-диалог, экспорт | +| 32 | `32-photo-trace-ai` | 25a, 29, 31a | v1.4.0. Фото-вырезы: альтернативный нейросетевой движок силуэтов (в браузере, веса по согласию, гибридный точный край), классический остаётся по умолчанию как baseline; сравнение движков | +| 33 | `33-keychain-ai-silhouette` | 32, 11, 29 | v1.5.0. Keychain Image: силуэт по ИИ (удаление фона на браузерной модели), те же ручки порога и сглаживания, один главный объект, результат хранится вместе с картинкой; яркость остаётся по умолчанию | +| 34 | `34-keychain-photo-relief` | 33, 32, 12, 17, 29 | v1.6.0. Keychain Image: рельеф из фото как смесь трёх источников формы (контур, глубина Depth Anything, детали — локальный контраст), кроп области (лицо), силуэт «весь кадр» (плашка), пресеты; результат хранится с картинкой | +| 35 | `35-smooth-outline` | 25a, 34, 29 | v1.7.0. Общий этап контура для вырезов (классика и ИИ) и Keychain: подгонка кубических кривых Безье с минимумом точек, острые углы сохраняются, зазор по кривой; стенки без ряби, детекция не меняется | +| 36 | `36-site-refresh` | 29 | v1.7.1. Актуализация сайта и README: тексты и теги карточек (без «скоро», «snap», «mask»), README по текущему продукту, превью Keychain из PoC-кольца с меньшим зумом, все превью Rack в 1 U, проверка отступов | +| 37 | `37-resource-monitor` | 31a, 32, 29 | v1.8.0. Вкладка «Ресурсы» в панели лога: честные показатели CPU/GPU/памяти для ресурсоёмких случаев (перестроение и отрисовка модели, ИИ на WASM и WebGPU) с пометкой «измерено / оценка / недоступно», отчёт по каждой задаче, режим диагностики (профилирование GPU); без FPS | +| 38 | `38-webmcp` | 31a, 29 | v1.9.0. Инструменты WebMCP для браузерных агентов: чтение и запись параметров активного режима (baseplate, box, rack, keychain shape) с валидацией, сводка модели, экспорт и сброс только с подтверждением пользователя, журнал и отмена действий агента, переключатель; без эффекта, если браузер не поддерживает WebMCP | ```mermaid gantt diff --git a/openspec/changes/31a-status-log/.openspec.yaml b/openspec/changes/31a-status-log/.openspec.yaml new file mode 100644 index 0000000..7cc41bb --- /dev/null +++ b/openspec/changes/31a-status-log/.openspec.yaml @@ -0,0 +1,2 @@ +schema: spec-driven +created: 2026-10-22 diff --git a/openspec/changes/31a-status-log/design.md b/openspec/changes/31a-status-log/design.md new file mode 100644 index 0000000..21f2a81 --- /dev/null +++ b/openspec/changes/31a-status-log/design.md @@ -0,0 +1,38 @@ +# Design + +## Context + +`MeasurementsBar` (in `features/viewport`) renders X, Y, Z of the model and, on the right, a spinner with "Generating…" when `busy` (from `useModel`, which sets it 200 ms after a request starts) or a check with "Ready". `Workspace` passes `busy` from each mode. There is no shared place to report anything else, and each feature shows its own banner. + +## Goals / Non-Goals + +**Goals:** +- One log for every part of the app, one place to look at it, the current line always visible. +- A small interface that features can call without knowing about the UI; the UI follows the interface language. + +**Non-Goals:** +- Persistence, telemetry, a debugger. + +## Decisions + +- **Module store, not React state.** `activityLog.ts` is a module-level store (array of entries, a list of jobs, subscribers) read by `useSyncExternalStore`, so workers' clients, hooks and plain functions can write to it without a provider, and logging never re-renders unrelated components. The panel and the bar subscribe to it; tests reset it between cases. +- **Entries carry keys, not sentences.** `{id, time, level, source, key, params, detail?, job?}`; the UI renders `t(key, params)`, so changing the language re-renders old lines. `detail` is raw technical text of an exception (message or short stack) shown inside a collapsed "Details" element on that line, never as UI text. Time is stored as a timestamp and shown as `HH:MM:SS.mmm`. +- **Jobs.** `startJob({source, key, params, cancel?})` returns `{update({key, params, progress}), done(key, params), fail(key, params, detail), cancelled()}`. Updates replace the job's current line (and progress) in place; the log keeps the job's start, the notable milestones (`done`/`fail`/`cancelled` and anything the producer logs explicitly) and does not store every progress tick. A job that runs more than 200 ms drives the status bar loader, which replaces the `busy` prop of `Workspace`. +- **Status bar.** Left: X, Y, Z as now. Right: with a running job, the spinner and the job's current line, plus a Cancel button if the job has one; with none, the latest entry for 4 s, then "Ready" (with the check) when a model exists; counters `⚠ n` and `✕ n` of the entries since the last open or clear. The whole right part is one button (`aria-label` "Open the log"): click or Enter/Space opens the panel. The line is an `aria-live="polite"` region, throttled to one announcement per second so a download does not flood a screen reader. At 390 px the X/Y/Z shorten and the line truncates with an ellipsis; the counters stay. +- **Panel.** `ActivityLogPanel` is rendered once by `AppShell` and opened through a tiny store (`openLog({source?})`), so any component, such as the model manager, can open it filtered to a source. Desktop: a resizable panel above the status bar covering about 40 % of the viewport area; below 768 px a bottom sheet over the parameter drawer. Content: a virtualised-free list (at most 500 lines), time, level icon, source label, text, optional Details; sticky filters (level: all / warnings / errors; source chips with counts); auto-scroll to the newest line that pauses when the user scrolls up and resumes with a "Jump to latest" button; Copy (plain text: app version, language, user agent, then the lines) and Clear with no confirmation (the log is disposable); Esc and a close button, focus returns to the status bar button. +- **Retention.** A ring buffer of 500 entries; the oldest drop first; running jobs are never dropped. Memory only: a reload starts a clean log. +- **Producers in this change.** Only what exists today: (1) model generation in `useModel` and `useKeychainModel` (a `generate` job per rebuild: start, duration and triangle count on done, the validation error key and the warnings of the `ModelResult` as warning lines, a failure as an error with detail); (2) image decoding, mesh and font import (name, size, downscaled, failure); (3) storage: IndexedDB or Cache API unavailable and the memory fallback, as a warning once per session; (4) the photo dialog: sheet found or not (confidence, paper), straightened, silhouettes found (count, sizes), cutouts created; (5) export: format, printer, plates, parts that overflow, file built. The AI changes add `ai` and `model` sources later. A slider move is not logged; only the rebuild it triggers is, and consecutive rebuilds within a debounce window update one job. +- **Keys.** Every message has `log..` keys in `en.json` and `ru.json`; the existing parity test covers them. +- **Privacy.** Entries never hold pixels, file contents or URLs of user files; names and sizes only; the Copy text states that it stays on the device. No network call is made by the log. +- **Compatibility.** The `busy` prop of `Workspace` and `MeasurementsBar` stays accepted for one change as an override that is ignored when jobs report; the modes switch to jobs in this change and the prop is removed in the same change (a task), since nothing outside the app uses it. + +## Risks / Trade-offs + +- [Log noise from frequent rebuilds] → jobs coalesce within the debounce window, progress updates are in place, a ring buffer bounds memory. +- [Screen reader flooding] → one polite announcement per second, only the status line is live, not the panel. +- [Duplicate sources of truth for "generating"] → the loader is derived from jobs only; tests check that no mode passes its own busy flag. +- [Technical detail shown in a non-translated language] → only inside "Details"; the line itself is a translated sentence. + +## Open Questions + +- Whether warnings from validation (`rack.error.*`, `baseplate.error.*`) should also keep their banners (yes for now: the banner is the call to action, the log is the record). diff --git a/openspec/changes/31a-status-log/proposal.md b/openspec/changes/31a-status-log/proposal.md new file mode 100644 index 0000000..2076959 --- /dev/null +++ b/openspec/changes/31a-status-log/proposal.md @@ -0,0 +1,29 @@ +# Proposal + +## Why + +The status bar under the 3D view (`MeasurementsBar`) shows the model's X, Y and Z and one indicator, "Generating…" or "Ready". That is all the app says about what it is doing. When a model does not appear, the reason is a banner or nothing; photo processing, image import, storage fallbacks and export have no visible trace; and the planned AI features (`32`–`34`) add long jobs (downloading a model, a run of several passes) that need a visible "what is happening now" and a place to read the whole story afterwards. + +Instead of a log hidden in one feature (the model manager), the status bar becomes a general, independent activity log of the whole app: one current line at a glance, and the full log one click away. + +## What Changes + +- **A shared activity log.** An app-level, memory-only log with levels (info, warning, error), a source (model generation, photo, image, storage, export, AI, app) and a text that follows the interface language. Long work is a *job* with a state, an optional progress and a Cancel; progress updates change the job's line in place instead of adding lines. +- **The status bar shows the activity.** X, Y and Z stay on the left. On the right a loader and the **current line** (the latest line of the running job, otherwise the last message for a few seconds, then "Ready"), with small counters of warnings and errors. Clicking it, or pressing Enter, opens the log. +- **The log panel.** Opens above the status bar (a bottom sheet on a phone): every line with time, level and source, filters by level and by source, Copy (plain text with the app version and language) and Clear, closes with Esc. It is reachable from anywhere that links to it, such as the "AI models" manager, not only from the status bar. +- **Existing work reports to it.** Model generation (start, duration, triangle count, validation errors and warnings with their messages, failures), image and font import, storage fallbacks (IndexedDB or Cache API unavailable), the photo dialog (sheet found, straightened, silhouettes found, cutouts created) and export (file built, plates, parts that do not fit). The AI changes (`32`–`34`) report through the same interface. +- **Not a telemetry.** The log lives in memory of the tab, holds names and sizes but never image data, and is never sent anywhere. + +## Capabilities + +### New Capabilities +- `activity-log`: the shared activity log, its status bar line and its panel. + +### Modified Capabilities + +## Impact + +- New `src/app/activityLog.ts` (store, `log.info/warn/error`, `startJob`, `useActivityLog` via `useSyncExternalStore`), `features/viewport/StatusBar.tsx` replacing the right part of `MeasurementsBar` (the X/Y/Z part stays), `app/ActivityLogPanel.tsx` rendered by `AppShell`; `useModel`, `useKeychainModel`, `imageStore`, `fontStore`, the photo dialog hook and the export panel emit events; the `busy` prop of `Workspace` is replaced by the job state; strings in en and ru; tests (the store, the line, the panel, producers, key parity, accessibility). +- Out of scope: persistence of the log across reloads, sending it anywhere or any telemetry, logging every slider move or render frame, a profiler, the AI events themselves (added by `32`–`34` through this interface), the Gridfinity or Rack calculations beyond their generation result. +- Version 1.3.0 (minor: a new capability). +- Depends on: `29-beta-label` (the versioning rule); `31-rack-shelf` is already archived. diff --git a/openspec/changes/31a-status-log/specs/activity-log/spec.md b/openspec/changes/31a-status-log/specs/activity-log/spec.md new file mode 100644 index 0000000..f70bf76 --- /dev/null +++ b/openspec/changes/31a-status-log/specs/activity-log/spec.md @@ -0,0 +1,116 @@ +# Spec Delta + +## Purpose +Defines the shared activity log of the app, the line that shows it in the status bar and the panel that shows all of it. + +## ADDED Requirements + +### Requirement: Shared activity log +The app SHALL keep one activity log for all its parts. Each entry SHALL have a time, a level (information, warning, error), a source and a message in the language of the interface, and MAY have technical details. Long work SHALL be a job with a state (running, done, failed, cancelled), an optional progress and an optional Cancel; progress updates SHALL change the job's line instead of adding lines. The log SHALL keep at most 500 entries, dropping the oldest first and never a running job, SHALL live in the memory of the page only, and SHALL NOT contain image data or file contents nor be sent anywhere. + +#### Scenario: Language follows the interface +- **WHEN** the interface language is changed +- **THEN** all lines, old and new, are shown in the new language + +#### Scenario: Progress in place +- **WHEN** a job reports its progress many times +- **THEN** its current line changes and the log does not gain a line for each report + +#### Scenario: Bounded +- **WHEN** more than 500 entries have been logged +- **THEN** the oldest finished entries are dropped and a running job stays + +#### Scenario: Not kept +- **WHEN** the page is reloaded +- **THEN** the log starts empty + +### Requirement: Status bar +The status bar under the 3D view SHALL keep showing the model's size, and SHALL show on its right a loader and the current line of the running job, with a Cancel when the job has one. When nothing runs it SHALL show the latest message for a few seconds and then that the model is ready. It SHALL show the number of warnings and errors, and the whole right part SHALL be one control that opens the log panel by click or by keyboard. A running job longer than a fraction of a second SHALL be what shows the loader, in place of any separate generating flag of a mode. + +#### Scenario: Current line +- **WHEN** a model is being generated or a photo is being processed +- **THEN** the status bar shows a loader and what is happening now + +#### Scenario: Ready +- **WHEN** no job is running and a model exists +- **THEN** the status bar shows that the model is ready + +#### Scenario: Counters +- **WHEN** a warning or an error has been logged +- **THEN** the status bar shows how many there are + +#### Scenario: Open +- **WHEN** the user clicks the right part of the status bar or presses Enter on it +- **THEN** the log panel opens + +#### Scenario: Cancel +- **WHEN** the running job has a Cancel and the user presses it +- **THEN** the job stops and its line says that it was cancelled + +#### Scenario: Narrow screen +- **WHEN** the screen is 390 px wide +- **THEN** the size shortens, the line is truncated with an ellipsis, and the counters and the loader stay visible + +### Requirement: Log panel +The log panel SHALL list every entry with its time, level, source and message, SHALL let the user filter by level and by source, SHALL keep the newest line in view until the user scrolls up and then offer to jump to the latest, SHALL show technical details of a line on request, and SHALL offer Copy (plain text starting with the app version, the language and the browser) and Clear. It SHALL close with a button and with Esc and return the focus to where it was opened. Other parts of the app, such as the model manager, SHALL be able to open it, also filtered to one source, from any page. + +#### Scenario: Filters +- **WHEN** the user chooses to show only errors, or only one source +- **THEN** only matching lines are shown, with the number of lines per source + +#### Scenario: Follow the newest +- **WHEN** new lines arrive and the user is at the end of the list +- **THEN** the list follows them, and when the user has scrolled up it stays and a control jumps to the latest + +#### Scenario: Copy +- **WHEN** the user presses Copy +- **THEN** the text of the shown lines, with the app version, the language and the browser, is put on the clipboard and the panel says so + +#### Scenario: Clear +- **WHEN** the user presses Clear +- **THEN** all finished entries are removed and the counters reset + +#### Scenario: Opened from elsewhere +- **WHEN** another part of the app opens the log for one source +- **THEN** the panel opens with that source selected, on any page, also where there is no status bar + +#### Scenario: Phone +- **WHEN** the screen is narrower than 768 px +- **THEN** the panel is a sheet from the bottom with the same content + +### Requirement: What the app reports +The app SHALL report to the log: each model generation (start, how long it took, the number of triangles, validation messages and warnings, failures with details), the import of images, meshes and fonts (name, size, whether the image was downscaled, failures), the fall-back to memory when the browser's storage is unavailable, the steps of the photo dialog (sheet found or not, paper, silhouettes found with their sizes, cutouts created) and the export (format, printer, plates, parts that do not fit, files built). Moving a control SHALL NOT add a line for each movement; only the rebuild it triggers SHALL, and rebuilds that follow each other within the delay of the model update SHALL be one job. + +#### Scenario: Generation +- **WHEN** a parameter changes and the model is rebuilt +- **THEN** one job reports the start and the result with its duration + +#### Scenario: Slider +- **WHEN** the user drags a slider +- **THEN** the log gains lines only for the rebuilds that actually run, not for every movement + +#### Scenario: Failure +- **WHEN** a generation fails or a parameter is invalid +- **THEN** an error or a warning line says why, with details on request, and the model that was shown stays + +#### Scenario: Storage +- **WHEN** persistent storage is unavailable +- **THEN** a warning says that results will not be kept, once per session + +### Requirement: Accessibility +The status line SHALL be announced politely to assistive technology at most about once a second, the control that opens the log SHALL have an accessible name, and the panel SHALL be reachable and closable with the keyboard. + +#### Scenario: Announcements +- **WHEN** a job reports progress many times a second +- **THEN** a screen reader announces at most about one line a second + +#### Scenario: Keyboard +- **WHEN** the user uses only the keyboard +- **THEN** they can open the log from the status bar, move through its controls and close it with Esc + +### Requirement: Language +All text of the status line, the panel and the messages the app logs SHALL be available in every language of the app. + +#### Scenario: Russian +- **WHEN** the interface language is Russian +- **THEN** the status line, the panel, its controls and the logged messages are in Russian diff --git a/openspec/changes/31a-status-log/tasks.md b/openspec/changes/31a-status-log/tasks.md new file mode 100644 index 0000000..811c3ba --- /dev/null +++ b/openspec/changes/31a-status-log/tasks.md @@ -0,0 +1,24 @@ +# Tasks + +## 1. Store and interface + +- [ ] 1.1 `app/activityLog.ts`: the store (entries, jobs, subscribers), `log.info/warn/error(source, key, params?, detail?)`, `startJob(...)` with `update / done / fail / cancelled`, a ring buffer of 500 entries that never drops a running job, `useActivityLog` via `useSyncExternalStore`, `openLog({source?})` and its state, a reset for tests. Tests: entries keep keys and params, progress updates change the job's line in place without adding lines, the buffer drops the oldest finished entries and keeps a running job, the loader state follows jobs longer than 200 ms, `cancelled` and `fail` set the state, subscribers are notified once per change. + +## 2. Status bar and panel + +- [ ] 2.1 `StatusBar` replacing the right part of `MeasurementsBar` (X, Y, Z stay): loader and the current line of the running job with Cancel, the latest message for 4 s then "Ready", counters of warnings and errors, one button that opens the log, a polite live region throttled to one announcement a second, truncation at 390 px; the `busy` prop of `Workspace` and the modes is removed in favour of jobs. Strings in `en.json` and `ru.json`. No artboard exists: built from `ui/` components and `theme.css` tokens, said in the commit; compare in en and ru and at 390 px. Component tests: the line follows a fake job, "Ready" after the message delay, counters, Enter and click open the panel, Cancel calls the job, the live region announces at most once a second, no mode passes its own busy flag. +- [ ] 2.2 `ActivityLogPanel` rendered by `AppShell`: list of entries with time, level, source and text, filters by level and by source with counts, follow and "Jump to latest", Details of a line, Copy (header with version, language, browser) and Clear, Esc and a close button with focus return, a bottom sheet below 768 px, openable from any page through `openLog({source})`. Component tests: filters, following and pausing on scroll, Copy writes the expected text, Clear resets the counters, Esc closes and returns the focus, opening with a source preselects it, the language switch re-renders old lines, key parity. + +## 3. Producers + +- [ ] 3.1 Model generation: `useModel` and `useKeychainModel` run one `generate` job per rebuild (rebuilds within the debounce are one job), report the duration and triangles on done, the validation key and warnings of the `ModelResult` as warning lines, a failure as an error with details. Tests: one job per debounced rebuild, the warning and error lines appear with their keys, a failed build keeps the previous mesh and logs the error. +- [ ] 3.2 Import, storage, photo and export: image, mesh and font import (name, size, downscaled, failure), the memory fallback of `imageStore`, `meshStore` and `fontStore` (a warning once per session), the photo dialog steps (sheet found with the confidence and paper, straightened, silhouettes found with sizes, cutouts created), export (format, printer, plates, overflow, files built). Tests: each producer emits its lines with the right source and level, the storage warning is logged once, no line holds pixels or file contents. + +## 4. Version + +- [ ] 4.1 Bump to 1.3.0: `package.json` and a `CHANGELOG.md` entry `1.3.0 — 31a-status-log`. Test: the existing version test passes with the new number. + +## 5. Verification + +- [ ] 5.1 `npm test` and `npm run build` are green, the dev server starts, no console errors. +- [ ] 5.2 In the browser, en and ru and at 390 px, in Box, Keychain and Cutout: the status bar shows "Generating…" lines while a model is built and "Ready" after, the log lists the generations with durations, an invalid parameter shows a warning with its message, importing an image and exporting a file are logged, the log opens by click and by keyboard, filters, Copy and Clear work, Esc returns the focus, a reload clears the log, nothing is sent to the network. diff --git a/openspec/changes/32-photo-trace-ai/.openspec.yaml b/openspec/changes/32-photo-trace-ai/.openspec.yaml new file mode 100644 index 0000000..b699b88 --- /dev/null +++ b/openspec/changes/32-photo-trace-ai/.openspec.yaml @@ -0,0 +1,2 @@ +schema: spec-driven +created: 2026-10-17 diff --git a/openspec/changes/32-photo-trace-ai/design.md b/openspec/changes/32-photo-trace-ai/design.md new file mode 100644 index 0000000..63bd316 --- /dev/null +++ b/openspec/changes/32-photo-trace-ai/design.md @@ -0,0 +1,65 @@ +# Design + +## Context + +Today `usePhotoTrace` builds `rectified = rectify(photo, corners, paper, rgb)` (4 px/mm, ≤ 2000 px) and `segmentation = segment(rectified, {margin, sensitivity})`. A `Segmentation` is `{width, height, scale, labels (Int32 per pixel, 0 = background), silhouettes[{id, areaMm2, widthMm, heightMm, bbox}], sensitivity, defaultSensitivity}`. Everything downstream (`traceSilhouette`, `tracedCutouts`, the silhouette list, click selection, the dialog's preview) reads only that structure. This is the seam: an AI engine has to produce a `Segmentation`, nothing else changes. + +Why the straightened sheet is a good input for a network: it is a top view at true scale with the table cropped away, so the network sees only paper and objects, the foreground is by construction "everything that is not the sheet", and mm ↔ pixel is known (4 px/mm). + +## Goals / Non-Goals + +**Goals:** +- An alternative engine behind the same `Segmentation`, selectable per session, default unchanged. +- The baseline is provably untouched and is the yardstick. +- No photo leaves the browser; weights are downloaded only after consent. +- A decision on the model backed by numbers on the fixtures. + +**Non-Goals:** +- Replacing or tuning the classic segmenter. Click prompts. Server inference. Non-commercial weights. + +## Decisions + +- **Engine as data, not as a fork.** `type SegEngine = 'classic' | 'ai'` lives in `PhotoSettings.engine` (default `'classic'`). `usePhotoTrace` keeps `segmentation` (classic, as now) and adds `aiSegmentation` computed only when the engine is `ai` or the compare view is on; the exposed `active` segmentation is chosen by the engine. `segment.ts`, `colour.ts`, `outline.ts` are not modified, only imported. Alternative: an `if` inside `segment()`. Rejected: it would touch the baseline and make its regression harder to prove. +- **Input and resolution.** The rectified RGB is resampled to the model's input (typically 512 or 1024 px for BiRefNet-lite, 320 for U²-Net-P), run, and the alpha is resampled back to the 4 px/mm raster (bilinear). A4 at 1024 px is ≈ 0.29 mm/px, so the network edge is only good to about ±0.5 mm: this is why the edge is refined (next point) and why the result is never used raw for the default. +- **Model choice by measurement (task 1).** The PoC (`poc/ai/`, results in `poc/ai/FINDINGS.md`) ran the candidates on the owner's photos: BiRefNet-lite at 1024 fails with `std::bad_alloc` on both backends and is dropped; RMBG-1.4 (non-commercial) gave the best result in ~6 s on WASM; ISNet loses a whole tool; U²-Net-P is fast but not better than the classic path; WebGPU brings no gain over WASM. Second round: InSPyReNet fp16 (MIT, WebGPU ~2 s, good), ormbg (Apache-2.0, WASM 15–20 s, good), ISNet imgly fp16 (MIT, WebGPU 0.25 s, misses part of the cutters), BiRefNet-lite 512 fp16 (MIT, 98 MB, good, WASM ~9 s, empty mask on WebGPU), U²-Net full (shadow leaks); MVANet and BEN2 mark the whole sheet and are dropped. The set offered in the manager comes from these results: InSPyReNet fp16 (WebGPU), BiRefNet-lite 512 fp16 and ormbg (WASM) as the recommendations, ISNet imgly fp16 as the fast option (its missed cutter parts are expected to come back with a lower threshold; to be confirmed on the PoC). A note: most networks lose part of the cutters, which the classic path finds, so the AI engine is an alternative for hard photos, not a better default. MODNet is portrait-only and excluded; `finegrain-box-segmenter` (396 MB) is excluded for size. The default AI engine uses the best candidate that reaches the targets in the table below; a second, smaller model may be offered as "lighter". If none reaches them the change stops after task 1 and records the numbers (the change is then archived as a negative result without a version bump, via `/opsx:update`). + + | Target on the two fixtures | Value | + |---|---| + | Tools found | 3 per photo, none merged or split | + | Size of each silhouette vs the baseline | within 1.0 mm (raw network), within 0.5 mm (hybrid) | + | Silhouette IoU vs the baseline | ≥ 0.97 (hybrid) | + | Time for one A4 on a laptop, WASM / WebGPU | ≤ 15 s / ≤ 3 s (informational) | + +- **Salient-object networks and shadows.** These networks were trained to cut out objects and usually leave a cast shadow out (the argument in `PREPROP_CUTOUT_GOOGLE.md`), but "usually" is not a guarantee, and thin dark parts (pliers jaws) can be eaten. The harness therefore reports, per fixture, the area difference to the baseline split into "network larger" (shadow leaked) and "network smaller" (part lost). +- **Mask → silhouettes.** Threshold the alpha at 0.5, then the same cleanup steps the classic path uses, imported from `geometry/raster` (`close`, `fillHoles`, `removeSpecks`, `labelComponents`, edge margin, `minSize`/`minArea` from `defaultSegmentParams`). The numbering rule is the classic one (top to bottom), so list order and colours are identical between engines for the same objects. Pure function `aiSegmentation(rectified, alpha, params) → Segmentation`, unit-tested on stored alphas. +- **Mask tuning (threshold and smoothing).** The network returns a soft alpha (0…1) with a soft zone along every edge. Two controls turn it into a silhouette, both applied to the cached alpha so the network is not re-run: a **threshold** (default 0.5, range 0.05…0.95; lower grows the outline into the soft zone, higher shrinks it) and a **smoothing radius** in mm (default 0.3, range 0…3; masked Gaussian from `geometry/raster/blur.ts` on the alpha before the threshold, σ = radius × scale, never below `MIN_SIGMA`). Smoothing evens out wavy and stair-stepped edges (the 1024 px output is stretched to 4 px/mm) and merges hairline cracks; too much removes thin parts, so the list shows each silhouette's size change and a hint appears when a silhouette shrinks by more than 1 mm. The PoC page already has both sliders with live recalculation, a greyscale view of the soft alpha, the outline length (shorter = smoother) and the area of the soft zone; the defaults and the size of the soft zone in mm are taken from its measurements. The two controls are user-facing and always visible under the AI engine (labels "Edge reach" / "Outline smoothness" with a one-line hint each and the numeric value; the technical names stay in code). Defaults are per model: the registry has `maskDefaults: {threshold, smooth}` for each entry (e.g. a lower threshold for ISNet, whose mask is tighter), found with the PoC sliders; switching the model loads them unless the user has touched the controls for the photo, and Reset returns to them. The values are session state of the dialog (`PhotoSettings.ai = {threshold, smooth}`), per photo not persisted, with a reset. With the hybrid edge the classic colour test places the final edge inside a band around this mask, so the two settings mainly decide which object parts count and how the band is placed; the traced outline is then smoothed by `traceSilhouette` as for the classic engine (σ = 1 px, 0.15 mm simplification). +- **Why the AI outline came out ribbed, and the fix.** The classic mask is made from the photo itself at 4 px/mm (0.25 mm per pixel) and `traceSilhouette` then blurs it (σ = 1 px), takes the 128 iso-line and simplifies it by 0.15 mm, so its outline is smooth. A model returns a fixed-size mask (512 × 512 for BiRefNet-lite) for the whole sheet, about 0.6 mm per output pixel, and stretching it with bilinear interpolation to 4 px/mm leaves a wobble with the period of the output pixel. Three measures, all measured in the PoC with a roughness metric (mean distance of the outline's vertices from the same outline smoothed with σ = 1.5 mm): (1) **bicubic (Catmull–Rom) upsampling** of the soft mask instead of bilinear; (2) a **second pass per object**: each object found by the first pass is cropped with a margin (12 mm) and run again, so the 512 px of the network cover the object only (about 0.3 mm instead of 0.6 mm per pixel), blended into the first result at the crop border; (3) the **vector outline smoothing** of the keychain PoC (resample, corner detection, Gaussian along the curve with corners fixed, default σ 0.25 mm) in place of the plain `simplifyLoop`. PoC result on the owner's flash photo: roughness of the classic outline 59 µm, AI with bicubic and vector smoothing 65 µm, AI with the second pass 58 µm, that is, the AI outline is as smooth as the classic one. The second pass costs one more model run per object (about 6 s each on WASM), so it is a toggle, on by default for the final result. +- **Hybrid edge.** For each silhouette: band = mask dilated by `band` mm (default 1.5) minus mask eroded by `band`. Inside the band the classic paper-colour deviation (`colourMask` from the paper surface fitted by `paperSurface`; `paperSurface` and `colourMask` are called, not changed) classifies pixels; the refined mask is `eroded core ∪ (band ∩ classic object pixels)`. If the classic test marks the whole band as paper or as object on a stretch longer than a few mm (the dark-part-in-shadow case), that stretch keeps the network edge. The refined labels go through the same `traceAlpha` as today. A setting `refine: on|off` (default on) exists in the compare view so the raw network edge can be inspected; it is not a persisted user option beyond the session. +- **Compare view.** `CompareOverlay` draws the other engine's silhouette outlines (already traced by `traceSilhouette`) as dashed lines over the straightened view, plus a table "tool: classic mm / AI mm / Δ". It costs one extra segmentation when open, and is the way the user, and the owner while developing, sees where the engines disagree. +- **ML runtime.** `src/ml/` is React-free. `matte(rgb, w, h) → Float32 alpha` is the only adapter this change needs. `ml.worker.ts` owns the inference session; `mlClient.ts` mirrors `modelClient.ts` (latest request wins, stale replies dropped); the worker is created lazily so classic users pay nothing, not even the library chunk (dynamic import). Backend: try WebGPU, fall back to WASM single-threaded (no COOP/COEP on Pages); report the backend in the model manager. +- **Model manager (a modal opened from the top bar).** `ModelManager` is a modal dialog opened by an "AI models" button in the `Header` (next to the language and unit switches), visible on every page and independent of any mode; the AI blocks of Cutout and Keychain only link to it ("Models…"). It has **sections by the `kind` of the registry entry**: "Silhouette (mask)" and "Depth", each listing its models with: name, what it is good for (fast / accurate), download size, licence, status (not downloaded / downloading with progress / ready / failed / unavailable on this device with the reason), and the buttons Download, Cancel, Delete; the footer of the modal shows the total space used, "Delete all" and a link that opens the shared activity log filtered to AI and model downloads (below). Several models can be downloaded at the same time as separate entries; the user picks the active one per kind (a radio in each section). Weights live on the client only: Cache API under a versioned key (revision hash), with a memory fallback and a notice when the browser refuses storage. The button in the `Header` shows a small indicator while a download or an AI run is in progress. +- **Explicit confirmation, no silent download.** There is one code path that starts a download, `requestDownload(model)`, and it always goes through a confirmation dialog in the manager (name, size, source host, licence; "Download" / "Cancel"); nothing prefetches, resumes in the background or downloads as a side effect of any other action. When the AI run button is pressed there is no download at all: **without a ready model of the needed kind the run button is disabled** and a hint under it says "Download a model to use AI" with a link that opens the manager; when the download finishes and a model is chosen, the button becomes active by itself (the state is derived from the model store, so it updates live). A test makes the downloader unreachable except through the confirmed path. +- **AI activity and the shared log.** The log itself is the shared activity log of `31a-status-log` (store, jobs, status bar line, log panel); this change only reports to it. Every phase of a model download and of an AI run is a job of source `model` (downloads) or `ai` (runs) with entries as i18n keys with parameters ("Downloading the model: 42 of 98 MB", "Starting the session (WASM)", "Object 2 of 3, second pass", "Building the outline"), progress updates change the line in place, `done` / `fail` / `cancelled` end it. `AiStatus` is a thin component that shows **only the current line of the feature's own job** with a spinner and Cancel next to the run button (photo dialog, Keychain silhouette, Keychain depth); the status bar shows the same line for any running job, and its panel shows the **full log** of the session, so the model manager has no log of its own: it only has a link "Open the log" that opens the shared panel filtered to the sources `model` and `ai`. A download started from the manager on a page without a status bar (a start page) is visible in its row of the manager and in the indicator of its Header button, and the log panel can be opened from the manager there. The log stays in memory and never leaves the browser. +- **Licences.** Only models under permissive licences (MIT, Apache-2.0, BSD, CC0) are offered. Non-commercial and copyleft weights are excluded: RMBG-1.4 (Bria, non-commercial) gave the best raw result in the PoC but is left out, because with the mask tuning below the permissive models reach a good result too (decision of the owner after the PoC), and the project stays free of licence caveats. The first download of any model shows its name, size, source host and licence and starts only after the click; the References page and the README list every model with its SPDX id and source. The PoC page keeps RMBG-1.4 marked `referenceOnly` as a quality yardstick; it is not part of the product registry. +- **Pinned weights.** The registry (`src/ml/models.ts`) holds per model: id, URL with a commit hash (not `main`), byte size, SHA-256 (verified after download), input size and normalisation, SPDX licence, source. A model enters the registry only with MIT / Apache-2.0 / BSD / CC0. The References page and README list them from this registry (a test checks every registry entry appears). +- **Backend per model.** The registry lists for each model the backends it works on and the preferred one, from the PoC: InSPyReNet fp16 and ISNet imgly fp16 are WebGPU-only (fp16 on WASM gives an empty mask or minutes), ormbg and U²-Net fail on WebGPU (`ceil_mode` in MaxPool) and run on WASM, RMBG-1.4 int8 runs on WASM faster than on WebGPU. The manager shows only models that the device can run (WebGPU and `shader-f16` detected at start) and marks the others as unavailable with the reason; a run that fails on the preferred backend is not silently retried on another unless the registry says it works there. +- **Mobile and weak devices.** The AI option is available at any width, but the dialog states the size, and on narrow screens or without WebGPU the lighter model is suggested first. The classic engine is always one click away and is what runs if the AI path fails (error banner, `active` falls back to classic). +- **Testing.** Unit tests never run a network. Frozen model outputs (alpha PNGs produced by the harness for each fixture and each chosen model, a few hundred KB) are committed under `geometry/photo/fixtures/ai/` and drive the post-processing tests. Component tests mock `mlClient`. The classic baseline has a new frozen-output test (silhouette count, bbox, area per fixture) next to the existing `photoFixtures.test.ts`, so any accidental change to it fails. +- **Rollout.** The engine switch is always visible, Classic selected. No flag. + +## Risks / Trade-offs + +- [The network does not beat the baseline on the owner's photos] → task 1 is a gate; the hybrid edge and the compare view are built only after it passes. +- [Network edge is "puffy" at 1024 px] → hybrid refinement; targets measured on the refined result. +- [Shadow leaks into the mask or a dark part is lost] → the band search clamps leaks to the classic edge where classic is reliable; lost parts are visible in the compare view and the user can switch engine. +- [100–300 MB download on mobile] → explicit consent with the size, the lighter model offered first on mobile, cancel, cache. +- [Library size and WASM files bloat the bundle] → dynamic import in the worker only on first AI use; the classic start-up path does not load them (checked in the build output size and the network panel). +- [Hugging Face availability, revision changes] → pinned hashes and SHA-256; on a failed download the error says which host failed and classic keeps working. +- [Single-threaded WASM is slow] → progress indicator, `computing` flag as for the classic path; measured in task 1, the lighter model is the WASM default. +- [Licence drift] → registry test refuses entries without an allowed SPDX id or without a pinned revision. + +## Open Questions + +- Is a request to huggingface.co acceptable at all (it reveals the IP), or must the weights be self-hosted (Cloudflare R2)? The design assumes huggingface.co with consent; self-hosting only changes the URLs in the registry. +- Which devices are the minimum target (an iPhone, a weak Android)? It decides whether the WASM path of BiRefNet-lite is shipped or only the light model. +- Click-to-select with SlimSAM / SAM 2.1-tiny is the strongest answer to touching and merged tools. It reuses the same runtime and the `Segmentation` seam and is proposed as the next change after the automatic path is measured. diff --git a/openspec/changes/32-photo-trace-ai/proposal.md b/openspec/changes/32-photo-trace-ai/proposal.md new file mode 100644 index 0000000..75d7277 --- /dev/null +++ b/openspec/changes/32-photo-trace-ai/proposal.md @@ -0,0 +1,36 @@ +# Proposal + +## Why + +The photo-to-cutout flow (`25`, `25a`) finds tools on a sheet of paper with a hand-tuned colour segmenter: a paper-colour model, a shadow-region growth and a hole fill. On the owner's two reference photos (`geometry/photo/fixtures/{flash,noflash}.jpg`) it works, but it is a pile of heuristics with known weak spots: dark parts that merge with their own shadow, objects close to the paper's colour, tools that touch and merge into one silhouette, glossy plastic and glass. Every new photo style needs new thresholds. + +Neural segmentation answers the question the heuristics approximate: "which pixels are the object". Models for salient-object matting and promptable segmentation run in the browser (ONNX Runtime Web / Transformers.js, WASM or WebGPU) and do not send the photo anywhere. The research is in `tmp/ai/BROWSER_AI_RESEARCH.md` (model catalogue) and `tmp/ai/BROWSER_AI_APPLICATION.md` (where to apply it); the preprocessing notes in `tmp/ai/PREPROP_CUTOUT_*.md` come to the same conclusion: use the network for "what", and keep a precise edge from classical code, because the requirement is 0.5 mm on size. + +The AI path is an **alternative**, not a replacement. The current segmenter stays exactly as it is and stays the default (the baseline): it works offline, needs no download and is the reference the AI result is measured against. + +## What Changes + +- **Engine choice on the silhouette step.** A control "Detection: Classic | AI" in the photo dialog. Classic is the default and is today's behaviour, untouched. AI is optional and needs a one-time model download with the user's consent. The choice is kept for the session with the rest of the dialog state. +- **AI silhouettes (automatic).** The straightened sheet (`rectify`) is run through a background-removal network; the foreground mask is cleaned (threshold, close, fill holes, edge margin, specks, connected components) into the same `Segmentation` the classic path produces, so selection, outline tracing, cutout creation and everything after it are shared. +- **Hybrid edge.** The network decides which pixels belong to an object; the exact edge comes from the classical code: a band of 1–2 mm around the network's mask is searched with the existing paper-colour test and sub-pixel `traceAlpha`. Where the classic test finds nothing in the band (a dark part merged with its shadow) the network's own edge is used. Result: the 0.5 mm scale accuracy of the baseline wherever the baseline is accurate, and the network's judgement where it is not. +- **Compare view.** A switch that shows the classic and the AI silhouettes over the straightened sheet (outline of the other engine as a dashed line), with the size of each silhouette in mm for both, so the user can pick the better result per photo. +- **ML runtime.** A separate worker (`ml.worker.ts`, not the Manifold worker), a client with "latest request wins" like `modelClient.ts`, a model manager, backend choice WebGPU → WASM, weights from a pinned Hugging Face revision. All UI text in en and ru. +- **Model manager (a modal of its own).** Opened by a button in the top bar of the app, always available and independent of any mode, so models can be downloaded or deleted at any time. Models are in sections by kind (silhouette mask, depth); each shows size, licence and status, with Download and Delete. A model is **never downloaded silently**: only after an explicit confirmation in the manager. The AI run button is disabled while no suitable model is ready, with a hint to download one, and becomes active when the download finishes. +- **AI activity and log.** While a model downloads or an AI run is going on, a loader and the **current line of the log** (what is happening now, for example "Downloading model: 42 of 98 MB", "Starting session (WASM)", "Object 2 of 3, second pass") are shown next to the run button, and every line also goes to the **shared activity log of the app** (`31a-status-log`): the status bar shows the same current line and opens the full log, so there is no separate log in the model manager (it only has a link to it). +- **Evaluation harness.** A spike script that runs the baseline and every candidate model on the fixtures and prints per-silhouette IoU and size error in mm; the model for the default AI engine is chosen from its output (task 1), not from the catalogue. + +## Capabilities + +### New Capabilities +- `cutout-photo-ai`: the optional neural silhouette engine for photo cutouts, the unchanged classic baseline, the hybrid edge, the compare view, and the model download and privacy rules. + +### Modified Capabilities + +## Impact + +- New `src/ml/` (React-free adapters `matte(image) → alpha`, runtime, model registry with pinned revisions and licences, cache), `ml.worker.ts` and `mlClient.ts`; new pure code `geometry/photo/ai.ts` (mask → `Segmentation`, hybrid edge band) next to `segment.ts`, which is **not edited**; `features/cutout/photo/` gets `EngineSwitch`, `ModelDialog`, `CompareOverlay` and `usePhotoTrace` gains the engine state and an AI segmentation next to `segmentation`; `app/` or settings gets the model manager entry; strings in en and ru; the References page lists every model with its SPDX licence and source; tests (post-processing on stored model outputs, baseline regression, component tests with a mocked `mlClient`). +- New dependency on an inference library (`onnxruntime-web` or `@huggingface/transformers`, chosen in task 1); weights are not committed and not bundled (100 MB limit of GitHub Pages), they are fetched on demand. +- The classic path's outputs on both fixtures stay byte-identical (checked by an existing test and a new frozen-output test). +- Out of scope: click-to-select / positive and negative points with a promptable model (SAM), a follow-up change once the automatic path is measured; depth maps, background removal for Keychain, prompt-to-parameters and other ideas from `BROWSER_AI_APPLICATION.md`; any server-side inference; non-commercial weights (RMBG-1.4/2.0, Depth Anything Base/Large) and AGPL code (Ultralytics); multi-threaded WASM (needs COOP/COEP headers that GitHub Pages does not send); offline pre-download of models; changing the classic segmenter. +- Version 1.4.0 (minor: a new option). +- Depends on: `25a-photo-trace-improvements`, `29-beta-label` (the versioning rule), `31a-status-log` (the shared activity log). diff --git a/openspec/changes/32-photo-trace-ai/specs/cutout-photo-ai/spec.md b/openspec/changes/32-photo-trace-ai/specs/cutout-photo-ai/spec.md new file mode 100644 index 0000000..0f17952 --- /dev/null +++ b/openspec/changes/32-photo-trace-ai/specs/cutout-photo-ai/spec.md @@ -0,0 +1,233 @@ +# Spec Delta + +## Purpose +Defines the optional neural silhouette engine for photo cutouts and the guarantee that the classic detection stays as the unchanged baseline. + +## ADDED Requirements + +### Requirement: Classic baseline unchanged +The classic silhouette detection described in `cutout-photo-trace` SHALL remain available, SHALL be the default engine, SHALL work without any download or network access, and SHALL produce exactly the same silhouettes as before this change for the same photo, corners, paper and settings. + +#### Scenario: Default engine +- **WHEN** the photo dialog is opened and a photo is chosen +- **THEN** the classic engine is selected and no model is requested + +#### Scenario: Unchanged result +- **WHEN** the two reference photos (with and without flash) are processed with the classic engine +- **THEN** each gives the same three silhouettes, with the same sizes and areas, as the recorded baseline + +#### Scenario: Offline +- **WHEN** the device has no network and the classic engine is used +- **THEN** detection works as before + +### Requirement: Engine choice +On the silhouette step the user SHALL be able to choose the detection engine, Classic or AI. The choice SHALL apply to the current photo and SHALL be kept while the dialog state lives (closing and reopening the dialog keeps it). Selection by click, the list with sizes, the sensitivity-independent steps (margin, depth, clearance) and the creation of cutouts SHALL work the same for both engines. + +#### Scenario: Switching engines +- **WHEN** the user switches from Classic to AI and the model is ready +- **THEN** the silhouettes are recomputed by the AI engine and shown in the same view and list, numbered top to bottom + +#### Scenario: Shared downstream +- **WHEN** silhouettes found by the AI engine are kept and Create is pressed +- **THEN** one cutout per kept silhouette is created exactly as for the classic engine, in true scale and position, with the bin footprint fitted + +#### Scenario: Sensitivity +- **WHEN** the AI engine is active +- **THEN** the sensitivity control, which belongs to the classic detector, is not offered for it, and the margin still applies + +### Requirement: Model manager +The app SHALL have a model manager, a modal window opened by a button in the top bar of the app, available on every page and independent of any mode. It SHALL list the available models in sections by their kind (silhouette mask, depth) with their name, purpose, size, licence and status, and SHALL let the user download, cancel, delete and choose the model used for each kind. Several models SHALL be downloadable and stored side by side. The models SHALL be kept in the browser's storage on the device; the window SHALL show the space they use and offer to delete all of them, and SHALL link to the app's activity log for the details of downloads and runs. + +#### Scenario: Always reachable +- **WHEN** the user is on any page of the app +- **THEN** the top bar has a button that opens the model manager, also when no AI feature is in use + +#### Scenario: Sections by kind +- **WHEN** the manager is opened +- **THEN** the models of each kind are in their own section, each with its size, licence and status + +#### Scenario: Download and delete +- **WHEN** the user downloads a model and later deletes it +- **THEN** progress is shown while downloading, the model is then ready for use, and after deleting its stored data is removed and it shows as not downloaded + +#### Scenario: Choice +- **WHEN** two models of one kind are downloaded +- **THEN** the user picks which one the AI engine uses for that kind + +#### Scenario: Activity indicator +- **WHEN** a download or an AI run is in progress and the manager is closed +- **THEN** the manager's button in the top bar shows that something is going on + +### Requirement: No AI run without a model +The button that starts an AI run SHALL be disabled while no suitable model is ready, and a hint next to it SHALL say that a model has to be downloaded and SHALL open the model manager. When the download finishes and a model is available, the button SHALL become active without any other action. A run SHALL NOT start a download. + +#### Scenario: Disabled without a model +- **WHEN** the AI engine is chosen and no silhouette model is downloaded +- **THEN** the run button is disabled and a hint offers to download a model, with a link to the manager + +#### Scenario: Active after the download +- **WHEN** the user downloads a model in the manager +- **THEN** when the download is complete the run button becomes active, with no other action + +#### Scenario: Deleting the model +- **WHEN** the user deletes the only model while the AI engine is chosen +- **THEN** the run button becomes disabled again, and an already computed result stays in use + +### Requirement: Model download and consent +A model SHALL be downloaded only after the user explicitly confirms in the model manager, and never as a side effect of another action, in the background or in advance. Before a download the app SHALL show its name, size, source host and licence, and the models offered SHALL be under permissive licences (MIT, Apache-2.0, BSD or CC0). Downloads SHALL show progress and be cancellable. If the download or the model fails, or the file's checksum does not match, the classic engine SHALL stay usable and the failure SHALL be shown. + +#### Scenario: Consent first +- **WHEN** the user starts the download of a model +- **THEN** a dialog states the size and that the source host sees the user's address, and nothing is downloaded until the user confirms + +#### Scenario: Never silent +- **WHEN** the user uses any feature of the app without confirming a download +- **THEN** no model file is requested from the network + +#### Scenario: Cached +- **WHEN** the model was downloaded earlier and the storage is intact +- **THEN** choosing it starts at once without a new download + +#### Scenario: Failure +- **WHEN** the download is interrupted or the checksum of the file does not match +- **THEN** an error says so, the file is not used, and the app stays on the classic engine + +#### Scenario: Storage unavailable +- **WHEN** the browser refuses persistent storage +- **THEN** the model works for the session and a notice says it will be downloaded again next time + +### Requirement: AI activity and log +While a model is downloading or an AI run is in progress, the app SHALL show a loader and the current line of the log, describing what is happening now, next to the control that started it, with a way to cancel. When the job ends the line SHALL show the result or the error. These lines SHALL be entries of the app's shared activity log, so the full log of downloads and runs, with the time of each line, is available in the log panel opened from the status bar, and the model manager SHALL link to it. The lines SHALL be in the language of the interface, and the log SHALL stay in the browser. + +#### Scenario: Current line +- **WHEN** an AI run goes through its phases +- **THEN** next to the run button a spinner and one line show the current phase, replacing the previous line as the phase changes + +#### Scenario: Download progress +- **WHEN** a model is downloading +- **THEN** the line shows the downloaded and total size, and the manager shows the progress of that model + +#### Scenario: Full log +- **WHEN** the user opens the activity log from the status bar, or from the link in the model manager, after a run +- **THEN** the lines of the download and of the run are listed with their time, and the log can be filtered to them + +#### Scenario: Error +- **WHEN** a run or a download fails +- **THEN** the line shows the error and the log keeps its details + +#### Scenario: Language +- **WHEN** the interface language is changed +- **THEN** the lines of the log are shown in the new language + +### Requirement: Privacy +The photo, the straightened image and the masks SHALL be processed in the browser only and SHALL NOT be sent anywhere by the AI engine. The only network traffic added by the AI engine SHALL be the download of the model files, from a fixed, versioned location, after consent. The user interface and the References page SHALL say so, and SHALL list every model with its licence and source. Only models under permissive licences (MIT, Apache-2.0, BSD or CC0) SHALL be offered. + +#### Scenario: No image upload +- **WHEN** a photo is processed by the AI engine +- **THEN** no network request carries image data + +#### Scenario: Licence list +- **WHEN** the References page is opened +- **THEN** each model used by the app is listed with its licence and source + +### Requirement: Silhouettes from the network +The AI engine SHALL run the straightened sheet through the model, turn its foreground mask into silhouettes with the same cleanup as the classic path (enclosed gaps filled, small specks and tiny silhouettes dropped, a margin along the paper edge ignored, one silhouette per separate object), and SHALL return them in the same form as the classic engine. Soft shadows beside a tool SHALL NOT be part of its silhouette. + +#### Scenario: Reference photos +- **WHEN** each of the two reference photos is processed by the AI engine +- **THEN** three silhouettes are found, and each tool's width and height are within 1 mm of the baseline's + +#### Scenario: Shadow +- **WHEN** a tool casts a soft shadow beside it +- **THEN** the silhouette follows the tool and not its shadow + +#### Scenario: Margin +- **WHEN** the margin is 3 mm +- **THEN** nothing within 3 mm of the paper edge is a silhouette + +#### Scenario: No objects +- **WHEN** the model finds nothing +- **THEN** the message for no silhouettes is shown and Create is disabled + +### Requirement: Mask tuning +With the AI engine the user SHALL be able to tune how the network's soft mask becomes a silhouette: a threshold (where inside the soft edge the outline is drawn, moving it outward or inward) and a smoothing radius in mm (how much the soft mask is blurred before the threshold, which evens out a jagged or wavy outline). Changing either SHALL update the silhouettes without running the network again, and the effect on each silhouette's size in mm SHALL be visible. Both controls SHALL be shown to the user on the silhouette step whenever the AI engine is active, with plain-language labels and a short hint each (the threshold as how far the outline reaches into the blurry edge of the object, the smoothing as how evenly the outline runs), not as technical names. Each model SHALL have its own default threshold and smoothing, since models differ in how sharp their mask is, and a reset SHALL restore the defaults of the model in use. Changing the model SHALL load that model's defaults unless the user has changed the controls for this photo. The soft mask SHALL be viewable as a greyscale image next to the silhouettes. + +#### Scenario: Live update +- **WHEN** the user moves the threshold or the smoothing slider +- **THEN** the silhouettes and their sizes update at once and the network is not run again + +#### Scenario: Threshold direction +- **WHEN** the threshold is lowered +- **THEN** the outlines grow outward into the soft edge, and when it is raised they shrink + +#### Scenario: Smoother outline +- **WHEN** the smoothing radius is raised +- **THEN** a wavy or jagged outline becomes smoother and the outline length shown does not grow + +#### Scenario: Thin parts +- **WHEN** the smoothing radius is large enough to erase a thin part of a tool +- **THEN** the size change is visible in the list and a hint says that a smaller radius keeps thin parts + +#### Scenario: Reset +- **WHEN** the user resets the mask settings +- **THEN** threshold and smoothing return to the defaults of the model in use + +#### Scenario: Per-model defaults +- **WHEN** the user switches from one model to another without touching the controls +- **THEN** the controls show the new model's defaults + +#### Scenario: Visible to the user +- **WHEN** the AI engine is active on the silhouette step +- **THEN** both controls are visible with their labels and hints, in the interface language + +### Requirement: Accurate edge +The AI engine SHALL place the silhouette edge by the paper-colour test within a narrow band around the network's mask and SHALL use the network's own edge only where that test does not separate the object from the paper. The sizes it reports SHALL stay within 0.5 mm of the classic result wherever the classic result is accurate. + +#### Scenario: Edge from the colour test +- **WHEN** a tool contrasts with the paper +- **THEN** its silhouette edge follows the colour edge, not the coarser edge of the network's mask + +#### Scenario: Dark part in shadow +- **WHEN** a dark part of a tool merges with its own shadow so that the colour test cannot find its edge +- **THEN** the network's edge is used along that stretch and the silhouette is still one piece + +#### Scenario: True scale +- **WHEN** a 40 × 20 mm rectangle lies on the paper and the corners are placed correctly +- **THEN** the AI silhouette measures 40 × 20 mm within 0.5 mm + +### Requirement: Smooth outline +The outline of a silhouette found by the AI engine SHALL be as smooth as the outline of the same object found by the classic engine: it SHALL NOT show a ribbed or wavy line that follows the resolution of the model's mask. The app SHALL upsample the model's soft mask smoothly, SHALL offer to refine each found object with a second model pass on its own crop, and SHALL smooth the traced outline along the curve while keeping sharp corners. + +#### Scenario: Comparable smoothness +- **WHEN** the same photo is processed by both engines +- **THEN** the roughness of the AI outline (mean distance of its vertices from the same outline smoothed by 1.5 mm) is within 25 % of the classic outline's + +#### Scenario: Second pass +- **WHEN** the second pass is on +- **THEN** each object is run again on its own crop and its outline is no rougher than without the second pass + +#### Scenario: Sharp tips +- **WHEN** a tool has a pointed tip +- **THEN** the smoothing does not round the tip away + +### Requirement: Compare engines +The user SHALL be able to compare the two engines on the straightened view: the silhouettes of the inactive engine are drawn as outlines over the view, and a table lists each tool's width and height for both engines and the difference. + +#### Scenario: Outlines +- **WHEN** compare is switched on +- **THEN** the outlines of the other engine appear as dashed lines over the straightened sheet + +#### Scenario: Sizes +- **WHEN** compare is on +- **THEN** each silhouette's size is shown for both engines with the difference in mm + +#### Scenario: Off by default +- **WHEN** the dialog is opened +- **THEN** compare is off and costs nothing + +### Requirement: Language +All text of the AI engine, the model dialog and the compare view SHALL be available in every language of the app. + +#### Scenario: Russian +- **WHEN** the interface language is Russian +- **THEN** the engine switch, the consent dialog, progress, errors and the compare table are in Russian diff --git a/openspec/changes/32-photo-trace-ai/tasks.md b/openspec/changes/32-photo-trace-ai/tasks.md new file mode 100644 index 0000000..781fc4c --- /dev/null +++ b/openspec/changes/32-photo-trace-ai/tasks.md @@ -0,0 +1,43 @@ +# Tasks + +## 1. Spike: do the models beat the baseline? (gate) + +- [x] 1.0 PoC page `poc/ai/` (`npm run poc:ai`): candidates run in the owner's browser on both photos; results in `poc/ai/FINDINGS.md`. BiRefNet-lite 1024 dropped (bad_alloc); round 2 dropped MVANet and BEN2 (mark the whole sheet). +- [ ] 1.1 Evaluation harness `scripts/eval-photo-ai.ts` (not shipped, not run by CI): for each fixture in `geometry/photo/fixtures` and for the owner's originals in `tmp/ai/photo_cutout_*.jpeg`, rectify, run the classic `segment` and each candidate model (BiRefNet-lite fp16, ISNet general, U²-Net-P; Node with `onnxruntime-node`), and print per silhouette: size in mm, IoU against the classic result, "network larger" and "network smaller" area, and time. Write each model's alpha for the fixtures to `geometry/photo/fixtures/ai/-.png`. Verify: the script prints the table for both fixtures. +- [ ] 1.2 Record the table and the decision in `design.md` (Decisions → Model choice): the default AI model, the lighter model, the pinned revisions and licences, the measured sizes of the files. Verify against the targets in `design.md`. If no model reaches the raw targets (3 tools, ≤ 1 mm) stop here, update the artifacts with `/opsx:update` and close the change as a negative result. + +## 2. Baseline guard + +- [ ] 2.1 Frozen-output test for the classic path (`photoBaseline.test.ts`): per fixture the silhouette count, each silhouette's bbox, area and size, recorded from the current code; fails on any change. Verify: passes on the current code before anything else is touched; `git diff` shows `segment.ts`, `colour.ts`, `outline.ts` untouched at the end of the change. + +## 3. Pure AI post-processing + +- [ ] 3.1 `geometry/photo/ai.ts`: `aiSegmentation(rectified, alpha, params) → Segmentation` (threshold, close, fill holes, margin, specks, components, numbering top to bottom, minimum size and area as the classic path). Tests on the stored alphas from 1.1: three silhouettes per fixture, sizes within 1 mm of the baseline, nothing in the edge margin, empty alpha gives no silhouettes, two touching blobs stay one silhouette. +- [ ] 3.1a Mask tuning in `geometry/photo/ai.ts`: `aiSegmentation(rectified, alpha, {threshold, smooth, …})` smooths the alpha (`gaussianBlur`, masked by the full raster) and thresholds it; the alpha is cached by the caller. Tests on the stored alphas: lowering the threshold never shrinks a silhouette and raising it never grows one (monotone), smoothing does not lengthen the outline, smoothing 0 equals the unsmoothed result, a thin synthetic part (1 mm) disappears at a large radius and survives at a small one, defaults give the sizes of 3.1. +- [ ] 3.1b Smooth AI outline: bicubic upsampling of the soft mask, the optional second pass per object (crop with a 12 mm margin, blended at the crop border), the corner-preserving vector smoothing, and a `roughness(outline)` helper. Tests on the stored alphas: the roughness of the AI outline of each fixture is within 25 % of the classic outline's, the second pass does not increase it, a synthetic spike keeps its tip within 0.3 mm, a bilinear upsampling of the same alpha is measurably rougher than the bicubic one. +- [ ] 3.2 Hybrid edge in `geometry/photo/ai.ts`: band around the mask, classic paper-colour test inside the band (calling `paperSurface` and `colourMask`), network edge kept where the test is uninformative, refined labels. Tests: sizes within 0.5 mm of the baseline and IoU ≥ 0.97 on both fixtures; a synthetic 40 × 20 mm rectangle (`synthetic.ts`) with a deliberately puffy mask measures 40 × 20 mm within 0.5 mm; a synthetic dark part merged into a shadow keeps the network edge and stays one piece; `refine: off` returns the raw mask result. + +## 4. ML runtime + +- [ ] 4.1 `src/ml/models.ts` registry (id, URL with commit hash, bytes, SHA-256, input size, normalisation, SPDX, supported and preferred backends, `maskDefaults` (threshold, smooth) found with the PoC, purpose, source) and a test that refuses entries without an allowed SPDX id (MIT / Apache-2.0 / BSD / CC0) or without a pinned revision. Verify: test green. +- [ ] 4.2 `src/ml/runtime.ts` and `ml.worker.ts`: lazy session creation by dynamic import, WebGPU then WASM single-thread, `matte(rgb, w, h) → Float32Array`, resize to the model input and back. `mlClient.ts`: latest request wins, stale replies dropped, progress and cancel. Tests with a fake worker for ordering and cancel; pre/post-processing tested on a tiny fixed array. Verify in the browser that a classic-only session loads no ML chunk (network panel and build output). +- [ ] 4.2a Capability detection (`src/ml/capabilities.ts`): WebGPU adapter and `shader-f16`, used by the manager to mark models as unavailable with a reason. Tests with a mocked `navigator.gpu`: no WebGPU hides WebGPU-only models, no `shader-f16` hides fp16 ones. +- [ ] 4.3 `src/ml/modelStore.ts`: download with progress and cancel, SHA-256 check, Cache API storage under the revision key, list, delete, `persistent` flag with memory fallback. Tests with a mocked `fetch` and cache, several models side by side: checksum mismatch rejects, delete removes, interrupted download leaves nothing, no storage falls back to memory. + +## 5. UI + +- [ ] 5.1 `usePhotoTrace`: `engine` in `PhotoSettings` (default `classic`), `aiSegmentation` computed only when needed, an `active` segmentation, AI error falls back to classic with an error state, `computing` covers the AI run. Tests: default is classic and calls no model; switching uses the mocked client; a failing client leaves classic active; closing and reopening keeps the engine. +- [ ] 5.2 `EngineSwitch` on the silhouette step (SegmentedControl Classic | AI, sensitivity hidden for AI) and `ModelManager` modal opened by an "AI models" button in the `Header` (always visible, independent of the mode; with an activity indicator), sections by registry `kind` (Silhouette mask, Depth), each row with name, purpose, size, licence, status (including unavailable with the reason), Download / Cancel / Delete, an active-model radio per kind, used space, "Delete all", an "Open the log" link to the shared activity log filtered to AI; a confirmation dialog before every download. The photo dialog links to it ("Models…"); the run button is disabled without a ready model, with the hint "Download a model to use AI" and a link, and becomes active by itself when the download ends. No artboard exists: built from `ui/` components and `theme.css` tokens, said in the commit; compare in en and ru and at 390 px. Component tests: consent before any download, progress, error banner, delete returns to "not downloaded" and asks for consent again, two models side by side, choosing the active one. +- [ ] 5.2b `AiStatus` (spinner, only the current line of the feature's own job, Cancel) and the reporting of downloads and runs to the shared activity log of `31a-status-log` (jobs of source `model` and `ai`, progress in place, done / fail / cancelled), with the strings of every event in `en.json` and `ru.json`. Tests: the line follows the events of a fake run in order and shows the result at the end, the activity log gets the start, the milestones and the end of each job, Cancel stops the job and logs it, errors appear in the line and in the log, switching the language re-renders the lines, key parity for every event key, nothing outside the manager calls the downloader, the run button follows the model store live (disabled, active after the mocked download, disabled after delete), the `Header` indicator appears while a job is active, the manager's link opens the log filtered to AI. +- [ ] 5.2a `MaskTuning` block, always visible under the AI engine on the silhouette step: two sliders with plain-language labels and hints ("Edge reach", "Outline smoothness"; threshold and smoothing in code) (smoothing is a length: `LengthField`), per-model defaults from the registry (`maskDefaults`), reset to them, greyscale view of the soft mask, size change per silhouette and the thin-part hint; updates without a model run (cached alpha). Component tests with a mocked `mlClient`: moving a slider changes the silhouette sizes and the mocked client is called once, reset restores the model's defaults, switching model loads its defaults unless the controls were touched, the hint appears when a silhouette shrinks by more than 1 mm, both languages. +- [ ] 5.3 `CompareOverlay` and the table of sizes (classic, AI, Δ) in the silhouette step; dashed outlines of the inactive engine over the straightened view; off by default. Tests: toggling draws the second outline set and the table values equal `silhouettes` of both engines; off costs no AI run. +- [ ] 5.4 Strings in `en.json` and `ru.json` (engine, consent, progress, errors, compare, References licence list); the References page and README list the registry models. Tests: key parity, a registry-to-References test, the `PhotoTips` and consent text exist in both languages. + +## 6. Version + +- [ ] 6.1 Bump to 1.4.0: `package.json` and a `CHANGELOG.md` entry `1.4.0 — 32-photo-trace-ai`. Test: the existing version test passes with the new number. + +## 7. Verification + +- [ ] 7.1 `npm test` and `npm run build` are green, the dev server starts, no console errors; `git diff main -- src/geometry/photo/segment.ts src/geometry/photo/colour.ts src/geometry/photo/outline.ts` is empty. +- [ ] 7.2 In the browser with the two reference photos, en and ru and at 390 px: Classic is the default and gives today's three silhouettes; AI asks for consent, downloads with progress, gives three silhouettes; compare shows both sets and sizes within the targets; deleting the model asks again; with the network blocked after the download AI still works and with it blocked before, the error appears and Classic still works; no request carries image data; cutouts created from AI silhouettes have the right size and position in the bin. diff --git a/openspec/changes/33-keychain-ai-silhouette/.openspec.yaml b/openspec/changes/33-keychain-ai-silhouette/.openspec.yaml new file mode 100644 index 0000000..5c881ba --- /dev/null +++ b/openspec/changes/33-keychain-ai-silhouette/.openspec.yaml @@ -0,0 +1,2 @@ +schema: spec-driven +created: 2026-10-18 diff --git a/openspec/changes/33-keychain-ai-silhouette/design.md b/openspec/changes/33-keychain-ai-silhouette/design.md new file mode 100644 index 0000000..d231fe7 --- /dev/null +++ b/openspec/changes/33-keychain-ai-silhouette/design.md @@ -0,0 +1,46 @@ +# Design + +## Context + +`decodeImage` returns `GrayImage {gray, alpha | null}`; the image is stored in IndexedDB and registered once in the worker (`imageRegistry`), params reference it by `imageId`. `buildSilhouette(img, s)` calls `toMask`, which uses `alpha ≥ 128` when the image has transparency, otherwise a brightness threshold; closing, area detection, specks and grow follow. `32-photo-trace-ai` provides `src/ml/` (worker, client, registry with pinned models, per-model backend and `maskDefaults`), the model manager and the mask-tuning maths (smoothing then threshold on a soft alpha). + +A keychain differs from a photo cutout in three ways that shape the design: the input is an arbitrary image (any aspect ratio, up to 4096 px) and not a rectified sheet; the result is one subject for one piece of printed plastic; and the silhouette is later sampled on a grid of 0.2–0.5 mm cells, so edge accuracy is far less critical than for a pocket that must fit a tool (a 50 mm keychain over a 1024 px matte is 0.05 mm per pixel). + +## Goals / Non-Goals + +**Goals:** +- Photo in, clean silhouette out, without leaving the app, with the same two controls and models as `32`. +- Brightness stays the default and unchanged; no image or mask leaves the browser. +- Reloading restores the same silhouette without running the network. + +**Non-Goals:** +- Editing the mask by brush, selecting objects by click, depth relief, Shape mode. + +## Decisions + +- **Soft matte as a stored plane, silhouette computed in the worker.** The main thread runs the network (the `32` client) and gets a soft alpha at the image's resolution (0–255, `Uint8Array`). It is registered once in the worker (`registerMatte`, registry `matteRegistry` like `textRegistry`) and params carry `silhouette.ai.matteId`. In `buildSilhouette`, when `source === 'ai'` and the matte exists, the image's alpha is replaced by `smoothThreshold(matte, ai.smooth, ai.threshold)` and the existing path (`toMask` alpha branch, close, areas, specks, grow) runs unchanged. Moving a slider therefore only rebuilds the silhouette in the worker, never re-runs the network, and the matte is not re-sent. Alternative: send a ready hard mask on each slider move. Rejected: re-registers megabytes per move and breaks the "params reference ids" convention. +- **Input to the network.** The whole image, long side capped at 2048 for inference (the stored image keeps its own size, ≤ 4096), fitted into the model's square input with the aspect ratio kept and the borders filled with the model's mean colour (the PoC "whole image" mode does exactly this); the output is cropped back and resampled bilinearly to the image size. The paper-sheet stretch used in `32` is not appropriate here. No hybrid edge: the resolution argument above makes it unnecessary, and the brightness image is not a reliable edge cue for photos. +- **Soft alpha semantics.** The matte keeps the network's probability, so the two controls mean the same as in `32`: threshold decides how far the outline reaches into the soft edge (default per model, 0.5 as a fallback), smoothing radius is a length in image pixels converted from mm of the keychain size (`LengthField`, mm of the final keychain, so the same value gives the same visual smoothness at any resolution). Defaults come from `maskDefaults` of the model. +- **Keep only the largest object.** A component filter after the threshold keeps the connected component with the largest area (8-connected, `labelComponents`) and drops the rest; enclosed gaps are untouched and become areas as before. Default on for AI, off for Brightness (where several shapes can be intentional, e.g. a logo with a separate dot). Stored as `ai.keepLargest`. +- **Automatic areas.** A photo of a ring handle, a bag strap or an animal with a gap between legs leaves enclosed background regions, and the owner's rule (PoC) is: large ones are meant to be open (a keyring hole must not be filled), small ones between strokes are noise and are filled. `SilhouetteParams.autoOpen` is an area threshold in mm² of the part (default **150**, 0 = off, i.e. today's behaviour of `fillHoles`); in `buildSilhouette`, after the areas are found and sized (pixels × the mm² of a pixel, from the part size), an area with no stored mode of its own becomes `open` if its size is at least the threshold and filled (`follow` with fill) if smaller. A mode the user sets on an area (open, level, follow) always wins, and stable ids are unchanged. Default on (150) for the AI source; Brightness keeps 0 so existing keychains are unchanged, and the Drawing preset of `34` sets it. The control is an area, so it is a `NumberField` labelled mm² and is not converted by the display unit (like the other unitless counters). +- **Persistence.** `imageStore` records get a second kind of entry, `matte`, keyed by `imageId` + `model` + `revision`, with the plane and its size; stored together with the original in the same DB (`gridfinity-keychain`), memory fallback with the existing `persistent` warning. The params store `source`, `ai.model`, `threshold`, `smooth`, `keepLargest`; on load the matte is read and registered. Deleting the model in the manager does not delete the matte; deleting or replacing the image does. If the matte is missing (cleared storage) the section shows "run again" and the silhouette falls back to Brightness until it is run, never to an empty mask. +- **Staleness.** The matte is tied to the image and the model: replacing the image drops it; choosing a different model keeps the old matte until the user runs the new one (shown as "matte from "), so a switch never silently re-runs a download-sized job. +- **Relation to existing alpha.** If the image already has transparency, the section shows the note "this image has transparency, it is used as is" with AI still selectable (the user may want a different cut), and the AI run ignores the image's alpha as input. SVG is rasterised by the existing path and is not offered AI. +- **UI.** `SilhouetteSection` gets `SegmentedControl` From: Brightness | AI. Under AI: model picker (from the manager, with a link to open it), the Remove-background button (disabled with the hint "Download a model to use AI" and a link to the manager while no model is ready, active as soon as one is) with the shared `AiStatus` next to it (spinner, the current line of the log, Cancel; the full log is in the shared activity log opened from the status bar), the shared `MaskTuning` block (edge reach, smoothness, per-model defaults, reset, soft-matte greyscale view, which can also open in the enlarged mask dialog), the keep-largest toggle and the hints. Brightness controls (threshold, invert) are hidden under AI because they have no effect there; close, fill holes, specks and grow stay, since they work on the resulting mask. The mask preview, areas and everything downstream read the resulting silhouette as today. +- **Hints.** After a run, if the image has more than 2 distinct dominant colours in the border band the hint is silent; if the image is nearly two-tone (a logo on a clean background: Otsu separability high, a border band of one colour) a hint says Brightness is usually better and quicker. Pure text, never blocks. +- **Performance.** The run is a few seconds to ~20 s depending on model and backend (see `FINDINGS.md`); the dialog shows progress, the panel stays usable, a new run cancels an old one (latest wins via the client). Huge images are capped for inference only. +- **Versioning of params.** `migrateKeychain` fills `source: 'brightness'` and the `ai` defaults for old records, so stored keychains are unchanged; with `source: 'brightness'` the pipeline output equals the current one (regression volumes in the existing tests). + +## Risks / Trade-offs + +- [The network picks the wrong subject or several subjects] → keep-largest handles stray bits; a different model or the controls help; click-to-select is a later change. +- [Fine details (fur, hair, thin chains) are lost or merged] → smoothing 0 and a lower threshold keep them; the grid cell limits the final detail anyway; the soft matte view shows what the network saw. +- [A 1024 px matte of a 4000 px photo is soft] → fine for a 0.2–0.5 mm grid on a keychain-sized part; the matte is upsampled bilinearly, not with fake detail. +- [Large downloads] → handled by the shared model manager and consent of `32`; a keychain run uses a model that may already be there from Cutout. +- [WebGPU-only models on devices without WebGPU] → the picker marks them unavailable with the reason (capability detection of `32`) and offers WASM models. +- [Storage growth] → a matte is about the size of the image's gray plane (≤ 16 MB at 4096²); stored once per image and model, removed with the image; memory fallback if storage is refused. + +## Open Questions + +- Whether the default model for Keychain should differ from the Cutout default (photos of animals and people versus tools on paper); the PoC "whole image" mode and a few pet and portrait photos from the owner decide it. +- Whether to offer a one-click "AI remove background" on import for images that look like photos (no transparency, busy border), or keep it manual. Starting manual. diff --git a/openspec/changes/33-keychain-ai-silhouette/proposal.md b/openspec/changes/33-keychain-ai-silhouette/proposal.md new file mode 100644 index 0000000..d96c004 --- /dev/null +++ b/openspec/changes/33-keychain-ai-silhouette/proposal.md @@ -0,0 +1,32 @@ +# Proposal + +## Why + +Keychain Image mode builds its silhouette from a threshold (Otsu or manual) on brightness, or from the alpha channel when the image has one. A logo on a white background works. A photo (a pet, a face, an object on a table) does not: brightness cannot tell the subject from the background, and the user fights the threshold, closing and areas without getting a clean outline. The usual answer is to remove the background first in some other tool and import a PNG with alpha. + +`32-photo-trace-ai` adds a browser-side neural engine for photo cutouts: an ML worker, a model manager, permissive-licence models and the two mask controls (threshold and smoothing). The same machinery produces exactly what Keychain needs, a soft alpha of the main subject, and the pipeline already has a place for it ("alpha wins"). This change connects them, so a photo becomes a keychain without leaving the app. + +## What Changes + +- **Silhouette source.** In Image mode the Silhouette section gets a choice "From: Brightness | AI". Brightness is today's behaviour, untouched and the default. AI removes the background with a model from the model manager of `32`. +- **Remove background (AI).** A button runs the network once on the whole image (aspect kept, padded to the model's input), shows progress, and stores the soft alpha ("matte") next to the image. The original image is kept; switching back to Brightness needs no new image. +- **Mask controls.** The same two user-facing controls as in `32` (edge reach = threshold, outline smoothness = smoothing radius) turn the stored soft matte into the silhouette. Changing them re-evaluates the mask without running the network. Defaults are per model (`maskDefaults` of the registry). +- **Keep the main object.** An option "Keep only the largest object" (on by default for AI) drops the stray bits a network sometimes leaves in the background; enclosed gaps stay and remain *areas* as before. +- **Persistence.** The matte is stored in IndexedDB with the image (`imageStore`), keyed by image, model and revision, so a reload restores the same silhouette without the network, and removing the model from the manager does not break an already computed matte. +- **Hints.** A transparent PNG keeps using its own alpha (AI is offered but not suggested); a logo on a clean background gets a hint that Brightness is usually better. SVG is not offered AI. +- **PoC.** `poc/ai/` has a "whole image (Keychain)" mode with padding, so models can be checked on keychain pictures before the work starts. + +## Capabilities + +### New Capabilities +- `keychain-ai-silhouette`: the AI silhouette source for Keychain Image mode, its mask controls, persistence and hints. + +### Modified Capabilities + +## Impact + +- `geometry/keychain.ts` (`SilhouetteParams` gets `source: 'brightness' | 'ai'` and `ai {model, threshold, smooth, keepLargest}`; `buildSilhouette` builds the alpha from the registered matte, then the unchanged pipeline), a matte registry in the worker next to `imageRegistry` (`ModelClient.registerMatte`, params carry `ai.matteId`), `features/keychain/` (`SilhouetteSection` source switch, `AiSilhouette` block, `useMatte`, `imageStore` matte records, migration normalising the new fields), reuse of `32`'s `src/ml/`, `ModelManager`, `MaskTuning`; strings in en and ru; tests. +- With `source: 'brightness'` the output equals the current one (regression volumes kept). +- Out of scope: Shape mode (no image), depth maps as a relief source, click-to-select or erase/restore brush on the mask, several subjects as separate keychains, background replacement or colour output, upscaling small images, any new model beyond the registry of `32`, Rack and other products. +- Version 1.5.0 (minor: a new option). +- Depends on: `32-photo-trace-ai` (ML runtime, model manager, mask tuning, registry), `11-keychain-relief`, `29-beta-label`. diff --git a/openspec/changes/33-keychain-ai-silhouette/specs/keychain-ai-silhouette/spec.md b/openspec/changes/33-keychain-ai-silhouette/specs/keychain-ai-silhouette/spec.md new file mode 100644 index 0000000..fa3c467 --- /dev/null +++ b/openspec/changes/33-keychain-ai-silhouette/specs/keychain-ai-silhouette/spec.md @@ -0,0 +1,149 @@ +# Spec Delta + +## Purpose +Defines the AI silhouette source for Keychain Image mode: removing the background of a photo with a model that runs in the browser, tuning the result and keeping it across reloads. + +## ADDED Requirements + +### Requirement: Silhouette source +In Image mode the user SHALL be able to choose where the silhouette comes from: Brightness (the existing threshold, invert and alpha behaviour) or AI (background removal by a model). Brightness SHALL remain the default and SHALL behave exactly as before. Choosing a source SHALL NOT change the image, and switching back and forth SHALL need no new import. + +#### Scenario: Default +- **WHEN** an image is imported +- **THEN** the source is Brightness and the silhouette and relief are the same as without this feature + +#### Scenario: Switching +- **WHEN** the user switches from AI to Brightness and back +- **THEN** each source shows its own silhouette from the same image, and an already computed AI result is not computed again + +#### Scenario: Controls of the source +- **WHEN** the source is AI +- **THEN** the brightness threshold and invert are not shown, and close gap, fill areas, remove specks and grow are still available + +### Requirement: Remove background with AI +With the AI source the user SHALL be able to run background removal on the whole image with a model chosen from the model manager. The run SHALL show a loader with the current line of the log (what is happening now) and be cancellable, SHALL work on any image aspect ratio without distorting it, and SHALL keep the original image. The image and the result SHALL stay in the browser. If no model is ready, the Remove background button SHALL be disabled with a hint to download a model (a link to the model manager), SHALL become active when a download finishes, and a run SHALL NOT download a model. + +#### Scenario: Run +- **WHEN** the user presses Remove background with a model ready +- **THEN** a loader and the current line of the log are shown next to the button, and on completion the silhouette preview and the relief show the subject without the background + +#### Scenario: Current line +- **WHEN** the run goes through its phases +- **THEN** the line shows what is happening now and changes as the phase changes, and shows the result or the error at the end + +#### Scenario: Any proportions +- **WHEN** the image is a wide landscape or a tall portrait +- **THEN** the subject's outline is not stretched compared with the image + +#### Scenario: No model +- **WHEN** no silhouette model is downloaded +- **THEN** the Remove background button is disabled and a hint offers to download a model with a link to the model manager, and when the download finishes the button becomes active with no other action + +#### Scenario: Cancel +- **WHEN** the user cancels a run or starts another +- **THEN** the cancelled run's result is discarded and does not replace the current silhouette + +#### Scenario: No upload +- **WHEN** a run is made +- **THEN** no network request carries image data + +### Requirement: Mask controls +With the AI source the user SHALL have the same two controls as for photo cutouts: edge reach (how far the outline reaches into the soft edge of the subject) and outline smoothness (a length in mm of the keychain). Changing either SHALL update the silhouette and the relief without running the model again. Each model SHALL have its own defaults, a reset SHALL restore them, and the soft result SHALL be viewable as a greyscale image. + +#### Scenario: Live +- **WHEN** the user moves edge reach or smoothness +- **THEN** the mask preview and the relief update and the model is not run again + +#### Scenario: Smoothness in mm +- **WHEN** the same keychain size and smoothness value are used on a small and a large image +- **THEN** the outline is smoothed by the same distance in mm on the part + +#### Scenario: Model defaults +- **WHEN** another model is chosen and its result is computed +- **THEN** the controls show that model's defaults unless the user has changed them for this image + +### Requirement: Keep the main object +With the AI source the user SHALL be able to keep only the largest connected object of the silhouette, on by default, so that stray fragments of the background are dropped. Enclosed gaps inside the kept object SHALL remain areas with their modes as before. + +#### Scenario: Stray fragments +- **WHEN** the model leaves a small speck of background away from the subject and the option is on +- **THEN** the speck is not part of the silhouette + +#### Scenario: Off +- **WHEN** the option is turned off +- **THEN** all objects the mask contains are kept + +#### Scenario: Areas +- **WHEN** the kept object has an enclosed gap such as a ring handle +- **THEN** it appears in the area list with a numeric id and the usual modes + +### Requirement: Automatic areas +Enclosed background regions of a traced silhouette SHALL be opened or filled automatically by size: regions at least as large as a threshold in square millimetres of the part SHALL be open, smaller ones SHALL be filled. The threshold SHALL be set by the user, with 0 meaning no automatic rule. A mode the user has set on a region SHALL always win. The default SHALL be 150 mm² for the AI source and 0 for Brightness, so existing keychains do not change. + +#### Scenario: Ring handle +- **WHEN** the AI silhouette has a large enclosed gap such as a key ring hole +- **THEN** it is open without the user marking it, and the area list shows it as open + +#### Scenario: Small gap +- **WHEN** the silhouette has a small enclosed gap smaller than the threshold +- **THEN** it is filled + +#### Scenario: User wins +- **WHEN** the user sets a mode on an area +- **THEN** that mode is kept whatever the threshold says, also after the threshold changes + +#### Scenario: Off +- **WHEN** the threshold is 0 +- **THEN** the areas follow the existing fill and open rules + +#### Scenario: Existing keychain +- **WHEN** a keychain saved before this feature is loaded +- **THEN** its areas behave as before + +### Requirement: Persistence of the AI result +The AI result SHALL be stored with the image in the browser's storage and restored after a reload together with the source, model and controls, without running the model again. Deleting the model from the model manager SHALL NOT remove an existing result. Replacing or removing the image SHALL remove its result. If the stored result is missing, the app SHALL say so, offer to run again and use Brightness meanwhile, never an empty silhouette. If storage is unavailable the AI source SHALL work for the session and the user SHALL be warned that the result will not persist. + +#### Scenario: Reload +- **WHEN** the user runs AI background removal, adjusts smoothness and reloads +- **THEN** the same silhouette is shown without running the model + +#### Scenario: Model deleted +- **WHEN** the user deletes the used model in the model manager +- **THEN** the silhouette from the stored result remains + +#### Scenario: Result lost +- **WHEN** the stored result is gone but the AI source is selected +- **THEN** a message offers to run again, and the silhouette follows Brightness until then + +#### Scenario: Storage unavailable +- **WHEN** persistent storage cannot be used +- **THEN** the AI result works in memory and a warning is shown + +### Requirement: Hints +The app SHALL tell the user when an image has its own transparency (it is used as is, AI can still be chosen) and when an image looks like a two-tone logo on a plain background (Brightness is usually quicker and sharper). SVG images SHALL NOT offer the AI source. Hints SHALL never block an action. + +#### Scenario: Transparent image +- **WHEN** a PNG with transparency is imported +- **THEN** a note says its transparency is used and the AI source remains selectable + +#### Scenario: Logo +- **WHEN** the image is a black logo on a white background +- **THEN** a hint suggests Brightness + +#### Scenario: SVG +- **WHEN** the image is an SVG +- **THEN** the AI source is not offered + +### Requirement: Compatibility +Keychains saved before this feature SHALL load with the Brightness source and unchanged settings, and with the Brightness source the relief SHALL be identical to what it was before this feature. + +#### Scenario: Old record +- **WHEN** a record without the new fields is loaded +- **THEN** it opens with the Brightness source and produces the same relief as before + +### Requirement: Language +All text of the AI silhouette source, its controls and hints SHALL be available in every language of the app. + +#### Scenario: Russian +- **WHEN** the interface language is Russian +- **THEN** the source choice, button, controls, progress, hints and warnings are in Russian diff --git a/openspec/changes/33-keychain-ai-silhouette/tasks.md b/openspec/changes/33-keychain-ai-silhouette/tasks.md new file mode 100644 index 0000000..2f5b946 --- /dev/null +++ b/openspec/changes/33-keychain-ai-silhouette/tasks.md @@ -0,0 +1,31 @@ +# Tasks + +## 1. Check on keychain pictures (spike) + +- [ ] 1.1 With `poc/ai/` in "whole image (Keychain)" mode, run the models of `32`'s shortlist on 6–10 pictures: a pet, a portrait, an object on a table, a plush toy, a logo with transparency, a two-tone logo, a wide landscape-format photo. Record per picture and model: subject found whole, stray fragments, edge quality at the threshold and smoothing that look best, time. Write the table and the per-model `maskDefaults` for keychain pictures into `design.md` (Open Questions) and `poc/ai/FINDINGS.md`. If a model cannot be used on keychain pictures say so and stop for `/opsx:update`. + +## 2. Geometry (React-free) + +- [ ] 2.1 `geometry/raster/matte.ts`: `smoothThreshold(matte, w, h, radiusPx, threshold) → Uint8Array alpha`, `keepLargest(mask, w, h)`. Tests: threshold monotone (lower never shrinks), radius 0 equals plain threshold, smoothing does not lengthen the outline, keepLargest drops a speck and keeps an enclosed gap, ties and empty mask are safe. +- [ ] 2.2 `SilhouetteParams.source` and `ai {model, threshold, smooth, keepLargest, matteId}`, `buildSilhouette` builds the alpha from the registered matte (`matteRegistry` in `geometry/model.ts`, `ModelClient.registerMatte`) and runs the unchanged pipeline; missing matte under `ai` falls back to Brightness and reports it in `SilhouetteResult`. Tests: with `source: 'brightness'` the existing regression volumes and bounding boxes are unchanged; a synthetic image plus matte gives the expected mask and the same relief as the equivalent alpha PNG; slider changes do not touch the registered matte; a missing matte reports the fallback. +- [ ] 2.2a Automatic areas: `SilhouetteParams.autoOpen` (mm², default 150 for `ai`, 0 otherwise) and its use in `buildSilhouette` (size of each area in mm² from the part size; explicit per-area modes win; stable ids unchanged). Tests: a ring gap above the threshold is open and a speck gap below it is filled, an explicit mode survives a threshold change, 0 reproduces the existing behaviour and volumes, the size in mm² does not depend on the image resolution, the threshold works for a Brightness silhouette when set. +- [ ] 2.3 `migrateKeychain` fills `source: 'brightness'` and the `ai` defaults for old records and normalises partial values. Tests: old record round-trip, partial and corrupt `ai` data fall back to defaults. + +## 3. Image side + +- [ ] 3.1 `features/keychain/matteStore.ts` (or `imageStore` records of kind `matte`): put/get/delete by `imageId` + model + revision in the existing DB, memory fallback with the `persistent` flag, removal with the image. Tests with `fake-indexeddb`: round trip, replacing the image drops its matte, deleting a model does not, storage refusal falls back to memory. +- [ ] 3.2 `features/keychain/matte.ts`: whole-image run through the `32` client (aspect kept, padded input, inference cap 2048, output cropped and resampled to the image size, progress, cancel, latest wins) and `useMatte` (state, restore on load, registration in the worker). Tests with a mocked client: padded input geometry for a wide and a tall image, output mapped back without distortion, cancel discards, a stale result does not replace a newer one, restore registers the stored matte without calling the client. + +## 4. UI + +- [ ] 4.1 `SilhouetteSection`: source switch Brightness | AI (hides brightness threshold and invert under AI), `AiSilhouette` block (model picker with a link to the manager, Remove background button (disabled with a hint and a link to the manager without a model, active by itself after the download) with `AiStatus` (loader, current line, Cancel), shared `MaskTuning` with per-model defaults and the soft-matte view, keep-largest toggle, Open-areas-larger-than field in mm² (`NumberField`, with a hint), lost-result message, hints for transparent, logo and SVG). Strings in `en.json` and `ru.json`. No artboard exists: built from `ui/` components and `theme.css` tokens, said in the commit; compare in en and ru and at 390 px. Component tests: Brightness default and unchanged, controls hide and show per source, the button is disabled with a hint and a manager link when no model is ready and becomes active when the model store reports a model, the loader and the current line follow the events of a fake run, sliders update the silhouette without a client call, keep-largest, the threshold field opens and fills areas and does not override a mode the user set, hints, SVG has no AI, lost result falls back with a message, key parity. +- [ ] 4.2 The enlarged mask dialog and `MaskPreview` show the AI silhouette and can switch to the soft matte view; areas, selection and everything downstream work on the AI silhouette. Tests: preview follows the source, the area list is built from the AI mask. + +## 5. Version + +- [ ] 5.1 Bump to 1.5.0: `package.json` and a `CHANGELOG.md` entry `1.5.0 — 33-keychain-ai-silhouette`. Test: the existing version test passes with the new number. + +## 6. Verification + +- [ ] 6.1 `npm test` and `npm run build` are green, the dev server starts, no console errors; with the Brightness source the existing keychain tests and volumes are unchanged. +- [ ] 6.2 In the browser, en and ru and at 390 px, with a pet photo, a portrait and a logo: AI run with progress, silhouette and relief appear, sliders update without a model run (network panel shows no model activity), keep-largest drops a stray bit, a ring-shaped gap is open by itself and a small gap is filled, reload restores the same relief without a run, deleting the model keeps the silhouette, switching to Brightness and back works, a transparent PNG and an SVG behave as described, the exported STL is one valid piece, no request carries image data. diff --git a/openspec/changes/34-keychain-photo-relief/.openspec.yaml b/openspec/changes/34-keychain-photo-relief/.openspec.yaml new file mode 100644 index 0000000..5c881ba --- /dev/null +++ b/openspec/changes/34-keychain-photo-relief/.openspec.yaml @@ -0,0 +1,2 @@ +schema: spec-driven +created: 2026-10-18 diff --git a/openspec/changes/34-keychain-photo-relief/design.md b/openspec/changes/34-keychain-photo-relief/design.md new file mode 100644 index 0000000..85a36fd --- /dev/null +++ b/openspec/changes/34-keychain-photo-relief/design.md @@ -0,0 +1,59 @@ +# Design + +## Context + +`SourceParams = {kind: 'distance' | 'brightness', contrast, gamma, invert, smooth, despeckle}`. In `buildGrid` the brightness source samples the gray plane at the grid nodes inside the silhouette and maps it with `brightnessValue` to heights `floor + value × (hmax − floor)`; then the rim ceiling, smoothing (masked Gaussian), profile (bevel, dome, terrace, flat, engraved), detail lines and patterns. `buildSilhouette` makes the mask from brightness or alpha; `33` adds the AI matte and its registry; `32` the ML runtime and model manager. + +## What the PoC showed (cat photo) + +Depth V2 Small gives a smooth depth map (the owner found it good as a form); on its own it does not carry the facial features. Local contrast of the brightness on a face crop (400 × 300 px of a 960 × 1280 photo) makes the eyes, nose, whiskers, stripes and ear clear, and 4 tone levels keep the face recognisable; on the whole frame the face is too small (`poc/ai/findings/`). The contour bevel is the existing form. The three sources therefore play different roles, and the PoC combiner is the tool to pick the mix. + +## Goals / Non-Goals + +**Goals:** +- A recognisable portrait relief from a photo by mixing contour, depth and detail, with controls the user understands; nothing changes for existing keychains. +- Everything stays in the browser. + +**Non-Goals:** +- Face detection, sculpting, line extraction, colour. + +## Decisions + +- **One mix, three shares.** In `buildGrid`, with `source.kind = 'photo'`, the height value at a node inside the silhouette is + `value = (wC·contour + wD·depth + wT·detail) / (wC + wD + wT)`, all three terms in 0…1, then the existing contrast, gamma, invert and smoothing, then `floor + value × (hmax − floor)`, then the rim ceiling, profile and patterns as for any source. `contour` is the existing bevel value (`reliefValue` of the distance to the edge, so wC alone equals the Distance source), `depth` the depth plane normalised inside the silhouette, `detail` the boosted brightness. A share is stored in 0…100 and normalised by the sum, so the user sets proportions and the relief always uses the full height range; sum zero falls back to Contour. Shares of unavailable sources (no depth computed yet) count as zero and the panel says so. +- **Why normalised shares and not an additive detail.** Three sliders that always add up to the whole are easy to read ("40 % depth, 30 % detail") and cannot overflow the range. Detail is zero-centred (mean 0.5), so raising its share changes structure, not the mean height. The bevel ceiling keeps the edge rolling off, so a high detail share never produces a cliff at the edge. +- **Depth plane like the matte of `33`.** The main thread runs the depth model through the `32` client and gets a plane at the image's resolution, normalised 0…1 (2nd–98th percentile; farther 0, nearer 1), registered once in the worker (`registerDepth`, `depthRegistry`); params carry `source.depthId`. In `buildGrid` it is sampled like brightness and renormalised over the silhouette pixels (robust min/max inside the mask) so a far background does not waste the range. Sliders only rebuild in the worker. +- **Depth model and input.** Depth Anything V2 Small ONNX (`onnx-community/depth-anything-v2-small-ONNX`, Apache-2.0): q4f16 20 MB, fp16 50 MB, weights in a separate `.onnx_data` file, so the runtime needs external-data support (path, bytes and SHA-256 of both files in the registry entry). Registry kind `depth` next to `matte`; the manager shows the kind and the picker filters by it. Input: the working image (the crop when set, long side capped at 2048), resized to a multiple of 14 with the aspect kept (518 for this model), output resampled bilinearly back. Depth Anything 3 Small (Apache-2.0, newer) is compared in the spike; Base and Large are non-commercial and not offered. +- **Depth and the crop.** Depth computed on the crop has more pixels on the face. The depth record stores the crop it was computed for; when the crop changes the depth is kept and resampled (stale but usable) and marked "computed for another crop, recompute for better detail"; the user decides to re-run (a few seconds). Contour and Detail follow the crop live. +- **Detail boost.** `localContrast(gray, w, h, radiusPx, amount)`: subtract a Gaussian-blurred copy, divide by the local standard deviation (blurred squared deviation with a floor against noise), `0.5 + 0.5·tanh(amount·v)`. The radius is stored in **mm of the part** (`boostRadius`, default ≈ 1.4 mm, found with the PoC at about 14 px on a 400 px face crop printed 40 mm wide) and converted to pixels from the crop and the keychain size, so the same value looks the same at any resolution. Amount 0 = off (detail is then a flat 0.5, the same as no detail). +- **Tone levels.** `levels` 0 = continuous (default), 2–8 = quantise the detail value, with `levelSoftness` (0 hard steps, 1 smooth ramps) so the steps are printable slopes; only the detail term is quantised, so contour and depth stay smooth. The Terraced profile still works on the final height. +- **Crop.** `crop = {x, y, w, h}` fractions of the image (default full). `buildSilhouette` and `buildGrid` read the planes (gray, alpha, matte, depth) through a cropped view made once per change in the worker; size, text and mount fractions stay relative to the silhouette box. `CropEditor` over the image: drag, corner and edge handles, aspect lock (free, 1:1, 4:3, 3:4), fit to subject from the AI mask box, reset; minimum 5 % of a side; live update of the preview; also in the enlarged mask dialog. Changing the crop never re-runs the matte model. +- **Whole frame silhouette.** `silhouette.source: 'frame'`: every pixel of the crop is material; threshold, close, areas and specks are hidden. With the Base plate (rounded rectangle, circle) this is a portrait plaque; the relief edge still rolls off through the bevel ceiling and rim. +- **Outline without steps.** Stairs on the edge come from three places, and each is removed: (1) a binary mask quantises the edge to a pixel, so the outline is taken from a continuous field instead (the soft matte of `33`, the grey level for Brightness, a blurred mask otherwise, iso-line at 0.5, sub-pixel); (the smoothing is the one that already works for cutouts: `traceSilhouette` blurs the mask with σ = 1 px at 4 px/mm, that is about 0.25 mm, takes the iso-line at 128 and simplifies it by 0.15 mm; the outline here reuses that routine or its parameters, expressed in mm of the part and exposed as an "Outline smoothness" value, default 0.25 mm, so thin spikes can keep sharper tips with a smaller value); (2) cutting the relief by whole grid cells leaves a staircase of cells, so the outline is traced as a vector polygon (`traceAlpha`, marching squares with sub-pixel interpolation, as for cutouts and text), simplified with a small tolerance (0.03–0.05 mm) and smoothed (Chaikin or a curvature-limited smoothing that keeps real corners such as spike tips), and the relief is clipped by that polygon exactly as the Shape mode already does with `shapeCrossSection` (extrude the polygon as a `CrossSection`, intersect with the relief, so the wall is exact and independent of the grid cell); (3) the grid only supplies heights, so its cell size no longer limits the outline. The PoC shows the effect with marching-squares clipping of the grid cells along the iso-line (`poc/ai/keychain.ts`): the outline is smooth to well under the cell size. Large enclosed areas stay open and small ones between strokes are filled through the existing areas of change 18 (`findHoles`), not a new rule. +- **Clean depth, printable slopes, final smoothing.** A monocular depth map is soft at the subject's edges and does not line up with the image, so the head "bleeds" into the background; and a sum of three sources can produce a wall steeper than a printer can build. Three steps, applied only to the Photo source: (1) **edge snapping**: the depth plane is passed through a guided filter (He et al.; guide = the grey image, box radius `snapRadius` in mm of the part, default 1 mm, regularisation `eps` derived from the user's 0–100 "Snap depth to edges" value, default 60, 0 = off), a linear-time operation of box filters that keeps depth steps where the image has an edge and flattens it where the image is flat; (2) **slope limit**: after the mix, contrast and gamma, the height field is relaxed so that no two neighbouring nodes differ by more than `tan(maxSlope) × cell`, by forward and backward sweeps (a Lipschitz clamp that only lowers peaks and keeps the base), default 50° (0 = off), not applied to `fixed` nodes (level areas) and never below the floor; (3) **final smoothing**: the existing masked Gaussian with a short radius (`finalSmooth`, default 0.2 mm, 0 = off) after the slope limit, so sweeps leave no creases. The order is depth → snap → mix → contrast and gamma → slope limit → final smoothing → bevel ceiling, profile, detail lines, patterns. Presets set the three values. The PoC combiner shows the effect (`poc/ai/keychain.ts`). +- **Detail as engraving, not bumps.** Mapping the tone of a photo straight to height turns fur and shading into random bumps. The default Detail mode is **Engrave**: only the darkest features (eyes, nose, mouth, stripes, whisker roots) are pressed in as grooves, `1 − groove` with `groove = smoothstep((dark − t) / (1 − t))`, `dark = 2·max(0, 0.5 − tone)` and a user "dark threshold" `t` (about 0.15 for a face crop), while mid-tones stay smooth; **Relief** (tone as height, the former behaviour) and **Both** remain as options. Grooves read well on a face at keychain scale and do not create noise on the body. The groove depth is the Detail share of the mix. +- **Corner-preserving outline.** The outline is a vector polygon traced from the soft field (resampled at ~0.7 px, corners found where the direction turns more than a threshold angle, default 50°, with local-maximum suppression), smoothed by a Gaussian along the curve with the corners held fixed (odd reflection between corners, ring smoothing when there are none), so curves become smooth while spike tips stay sharp; the mesh edge is cut exactly along it (signed distance to the polygon near the edge, marching squares on the cells). The PoC (`poc/ai/outline.ts`) implements this and shows crisp spike tips on the line-art samples. +- **Subject presets (a visible choice).** A "Subject" selector with **Face**, **Whole animal** and **Drawing / logo** sets what the three Photo shares do not: the silhouette source and the detail mode with its dark threshold, the part size, and the outline. Face: AI silhouette, crop to the face, Engrave with dark threshold 0.15; Whole animal: AI silhouette, whole frame crop, Engrave with threshold 0.18, no tone levels; Drawing / logo: Brightness silhouette, contour only, 60 mm, bevel 1.2 mm, outline smoothness 0.15 mm, automatic area opening at 150 mm² (`33`). An "Choose automatically" switch (on for a new image) picks one from the picture (few tones and thin strokes → Drawing; a model-found subject whose box fills the frame → Whole animal; otherwise Face) and turns itself off as soon as the user edits any value, which shows Custom. Each choice is a data record next to the profile presets, and `presetOf` returns Custom when values differ. These are separate from the intensity presets (Subtle, Portrait, Deep). +- **Presets.** Subtle / Portrait / Deep set the three shares, `hmax − floor`, boost radius and amount and levels (Portrait: about 30 / 40 / 30, 2 mm range, radius 1.4 mm, 4 soft levels; to be confirmed with the spike) and show Custom when edited; data next to the profile presets. +- **Heights and printing.** A height field grows from a flat back (existing design), prints face up without supports. Detail finer than the grid cell is lost: `quality` and the resolution warnings apply, with a hint when the cell is larger than about a third of the boost radius. +- **Persistence and migration.** Depth planes are stored in `imageStore` as `depth` records keyed by image, model, revision and crop, restored on load, kept when the model is deleted, dropped with the image, memory fallback with the `persistent` warning. `crop`, mix, boost and levels are keychain params (`migrateKeychain` fills defaults: full crop, `kind` unchanged, boost 0, levels 0). If the stored depth is missing the Depth share counts as zero with a "run again" message, never an empty relief. +- **Availability.** The Photo source appears only in Image mode, not for SVG images, and Depth only when a depth model exists; Contour and Detail work without any model. +- **Outline controls.** Under the Image section: Outline smoothness (`LengthField`, default 0.25 mm) and a collapsed Advanced group with Corner angle (degrees, default 50) and Sampling step (`LengthField`, default 1 mm); the fit tolerance stays at its 0.25 mm default here and `35` adds it with the original-polyline option for both products. The block is one component (`OutlineControls`) that `35` reuses in the photo dialog. +- **UI.** Relief source: Distance | Brightness | Photo. Under Photo: three share sliders with the mix as a stacked bar, the model picker and Compute depth (disabled with a hint and a manager link without a depth model, active after the download; the shared `AiStatus` with a loader, the current line of the log and Cancel; "for another crop" note), Detail boost (amount, radius as `LengthField`), Tone levels (count, softness), presets (Subtle / Portrait / Deep with Custom), the Subject selector (Face, Whole animal, Drawing / logo, with Choose automatically), Snap depth to edges, Max slope (degrees) and Final smoothing (`LengthField`), the existing contrast, gamma, invert and smoothing, and a height greyscale view next to the mask preview. `CropEditor` in the Image section; the Silhouette source gains "Whole frame". Strings in en and ru. +- **Performance.** The depth run is a few seconds and cancellable (latest wins); everything else is arithmetic in the worker. + +## Risks / Trade-offs + +- [Depth is soft or wrong at fur and at the subject's edge] → the mask clips the edge, the bevel ceiling hides the cliff, Detail adds the fine structure; the shares let the user lean on whichever source works for the photo. +- [Local contrast amplifies noise and JPEG artefacts] → floor in the divisor, despeckle and smoothing, the radius control. +- [A face is too small in the part to read] → the crop, the hint and the quality warning; a keychain-sized part cannot carry hair, the portrait is about eyes, nose and stripes. +- [A second model on top of the matte model] → shared manager and consent of `32`; both small, results are stored. +- [External-data ONNX needs runtime support] → registry and runtime tasks of this change (tested), PoC already loads it. +- [Quantised levels make jagged edges] → softness and the existing smoothing; hard steps are a choice. +- [Three shares are many controls] → presets first; the shares collapse into one stacked bar; a single "Photo" source keeps the panel short. + +## Open Questions + +- Defaults for the shares, radius, amount, levels and range (the spike picks them with the PoC combiner and real keychains). +- Whether depth on the face crop is clearly better than on the whole frame (`cat_face` sample in the PoC); decides if the crop-linked recompute prompt is shown by default. +- Whether to add a face detector for an automatic crop later. diff --git a/openspec/changes/34-keychain-photo-relief/proposal.md b/openspec/changes/34-keychain-photo-relief/proposal.md new file mode 100644 index 0000000..f3e7fba --- /dev/null +++ b/openspec/changes/34-keychain-photo-relief/proposal.md @@ -0,0 +1,37 @@ +# Proposal + +## Why + +With `33-keychain-ai-silhouette` a photo gives a clean outline, but inside the outline the keychain is still a plain bevelled plate. A portrait of a cat (or a person, or a pet's face) is only recognisable by what is inside the outline: the eyes, the nose, the ears, the fur pattern. The owner tried this on a cat photo: the outline is good, the face cannot be seen on the keychain. + +No single source of height does it. On the owner's cat photo (`poc/ai/FINDINGS.md`, images in `poc/ai/findings/`): + +- **Contour** (the existing bevel from the silhouette edge) gives the body of the keychain and a clean edge, but is the same for every photo. +- **Depth** (Depth Anything V2 Small) gives the overall form: the head raised over the body, the muzzle over the cheeks. It is smooth and soft on fine features. +- **Detail** (local contrast of the brightness, on a crop of the face) gives exactly the features depth smooths out: the eyes, the nose, the whiskers, the forehead stripes, the ear; on the whole frame the face is too small for it, on a face crop it reads well and four tone levels still keep the face recognisable. + +The three complement each other, so the relief is built as a **mix of the three**, with a share for each that the user controls, plus a crop so that the face has enough pixels. + +## What Changes + +- **Form mix.** The Relief source gets "Photo" next to Distance and Brightness. Under it three shares (Contour, Depth, Detail, in percent, normalised) are mixed into one height field; presets Subtle, Portrait and Deep set the shares, the height range and the detail settings, and edit to Custom. A share at zero switches that source off, and Contour alone is today's Distance source. +- **Depth.** Computed once by a model of kind `depth` from the model manager of `32` (Depth Anything V2 Small, Apache-2.0, 20–50 MB), stored with the image, normalised inside the silhouette so the background does not use up the height range. +- **Detail.** "Detail boost" (local contrast, radius in mm of the part so it means the same at any resolution) and optional "Tone levels" (2–8, soft or hard steps). Detail is zero-centred, so it adds structure around the form and does not lift the part by the photo's mean brightness. +- **Crop.** A non-destructive crop rectangle over the image (aspect lock, "fit to subject" from the AI mask box, reset). Everything after it works on the crop; the AI matte is cropped without re-running the network. Depth can be recomputed for the crop (more pixels on the face); a depth computed for another crop stays usable and is marked as such. +- **Whole frame silhouette.** A third silhouette source, "Whole frame", makes the crop a plaque (shape from the Base section) for a face portrait; the AI matte of `33` stays for cut-out heads and whole animals. +- **PoC.** `poc/ai/` has a "shape combiner": it takes the contour from a matte run, the depth from a depth run and the brightness of the same image, and shows the mixed height and a shaded relief with sliders for the three shares, the detail radius and levels, so the combination can be chosen before the work starts. + +## Capabilities + +### New Capabilities +- `keychain-photo-relief`: a relief built from contour, depth and detail, with crop and a whole-frame plaque. + +### Modified Capabilities + +## Impact + +- `geometry/keychain.ts` (`KeychainParams.crop {x, y, w, h}` fractions applied to the image planes before the silhouette; `silhouette.source` gets `'frame'` (extends `33`); `SourceParams.kind` gets `'photo'` with `mix {contour, depth, detail}`, `depthId`, `boost`, `boostRadius` (mm), `levels`, `levelSoftness`; `registerDepth` and `depthRegistry` in the worker; `buildGrid` samples the planes through the brightness path), `geometry/raster/` (`localContrast`, `quantise`, `normaliseDepth`), `src/ml/` registry gets `kind: 'depth'` with external-data support, `features/keychain/` (`CropEditor`, `PhotoReliefBlock`, `useDepth`, depth records in `imageStore`, migration), strings in en and ru, tests. +- With the Distance or Brightness source, no crop, a traced silhouette, boost 0 and levels 0, the output equals the current one. +- Out of scope: real metric depth or 3D reconstruction, face detection for an automatic crop, line or edge extraction, sculpt or brush editing of the relief, multi-colour output, larger non-permissive depth models, Shape mode. +- Version 1.6.0 (minor: new options). +- Depends on: `33-keychain-ai-silhouette`, `32-photo-trace-ai`, `12-keychain-patterns`, `17-keychain-smoothing`, `29-beta-label`. diff --git a/openspec/changes/34-keychain-photo-relief/specs/keychain-photo-relief/spec.md b/openspec/changes/34-keychain-photo-relief/specs/keychain-photo-relief/spec.md new file mode 100644 index 0000000..08657aa --- /dev/null +++ b/openspec/changes/34-keychain-photo-relief/specs/keychain-photo-relief/spec.md @@ -0,0 +1,236 @@ +# Spec Delta + +## Purpose +Defines how a photo becomes a recognisable portrait relief on a keychain by mixing three sources of form, the contour, the depth and the detail of the photo, with a crop and an optional whole-frame plaque. + +## ADDED Requirements + +### Requirement: Photo relief source +In Image mode the user SHALL be able to choose Photo as the relief height source, next to Distance and Brightness. The Photo source SHALL build the height inside the silhouette from three sources of form, each with a share the user sets: Contour (the distance from the silhouette edge, as in the Distance source), Depth (the depth of the photo computed by a model) and Detail (the local contrast of the photo's brightness). The shares SHALL be treated as proportions, so that the relief always uses the full height range. A share of zero SHALL switch that source off, and Contour alone SHALL equal the Distance source. + +#### Scenario: Mix +- **WHEN** the shares are Contour 30, Depth 40 and Detail 30 +- **THEN** the relief shows the keychain's edge roll-off, the overall form of the subject and the fine structure together + +#### Scenario: Contour only +- **WHEN** Depth and Detail are zero +- **THEN** the relief is the same as with the Distance source + +#### Scenario: Unavailable source +- **WHEN** no depth has been computed and the Depth share is above zero +- **THEN** the relief is built from the other sources, and the panel says that depth is missing + +#### Scenario: Proportions +- **WHEN** all shares are doubled +- **THEN** the relief is unchanged + +#### Scenario: Live +- **WHEN** a share changes +- **THEN** the preview and the relief update without running any model + +### Requirement: Depth source +The depth of the photo SHALL be computed once from the image, or from its crop, by a model of the depth kind chosen in the model manager and run in the browser, and kept with the image. Parts of the subject nearer to the camera SHALL be higher than parts farther from it. The depth range used SHALL be taken from the pixels inside the silhouette, so a far background does not use up the height range. The image and the result SHALL stay in the browser. While the depth is computed the app SHALL show a loader and the current line of the log next to the button, with a way to cancel. If no depth model is ready, the Compute depth button SHALL be disabled with a hint to download a model (a link to the model manager, depth section), SHALL become active when a download finishes, and a run SHALL NOT download a model. + +#### Scenario: Form +- **WHEN** the depth of a photo of an animal is computed and the Depth share is raised +- **THEN** the head and the front of the muzzle are higher than the surroundings of the face, inside the silhouette + +#### Scenario: Range inside the subject +- **WHEN** the background is far away and the subject is shallow +- **THEN** the subject's relief still spans the configured height range + +#### Scenario: No model +- **WHEN** no depth model is downloaded +- **THEN** the Compute depth button is disabled with a hint and a link to the model manager, and becomes active when the download finishes + +#### Scenario: Current line +- **WHEN** the depth is being computed +- **THEN** a loader and the current line of the log are shown next to the button, with the result or the error at the end + +#### Scenario: Crop +- **WHEN** the crop has changed since the depth was computed +- **THEN** the depth stays in use and is marked as computed for another crop, and the user can recompute it for the current crop + +#### Scenario: No upload +- **WHEN** the depth is computed +- **THEN** no network request carries image data + +### Requirement: Detail +The Detail source SHALL even out the lighting of the photo locally, so that fine structure such as eyes, a nose and stripes becomes visible as relief even in a flat-lit photo. Its radius SHALL be set as a length in mm of the finished part, so the same value gives the same result at any image resolution and size. Detail SHALL add structure around the form and SHALL NOT raise or lower the part as a whole because a photo is brighter or darker. The user SHALL be able to reduce the detail to a chosen number of levels (2 to 8), with a softness from hard steps to smooth ramps; zero levels SHALL mean continuous tones. Detail at zero amount SHALL add no structure. + +#### Scenario: Face detail +- **WHEN** the Detail share is raised on a face crop +- **THEN** the eyes, the nose and the stripes appear as relief around the face form + +#### Scenario: Radius in mm +- **WHEN** the same photo is used at two resolutions with the same radius in mm +- **THEN** the detail appears at the same scale on the part + +#### Scenario: Brightness +- **WHEN** a photo is uniformly brighter or darker +- **THEN** the height of the part does not change because of it + +#### Scenario: Four levels +- **WHEN** the levels are set to 4 +- **THEN** the detail uses four distinct heights, and the face stays recognisable + +#### Scenario: Softness +- **WHEN** the softness is raised +- **THEN** the steps between levels become gradual slopes + +### Requirement: Crop +In Image mode the user SHALL be able to crop the image to a rectangle by dragging it over the image, with handles on the corners and edges, an aspect ratio lock, a button that fits the crop to the subject when an AI mask exists, and a reset. The crop SHALL be non-destructive (the image stays whole) and SHALL apply to the silhouette, the relief, the preview and everything built from them. Changing the crop SHALL NOT run a matte model again. + +#### Scenario: Crop to a face +- **WHEN** the user drags the crop over the face in a photo +- **THEN** the mask preview and the relief show only that region, with the face filling the keychain + +#### Scenario: Non-destructive +- **WHEN** the crop is changed or reset +- **THEN** the original image is unchanged and resetting restores the full frame + +#### Scenario: Fit to subject +- **WHEN** an AI mask exists and the user presses Fit to subject +- **THEN** the crop becomes the mask's bounding box with a small margin + +#### Scenario: Aspect lock +- **WHEN** an aspect ratio is locked and a handle is dragged +- **THEN** the crop keeps that ratio + +#### Scenario: No model run +- **WHEN** the crop changes and the silhouette comes from an AI mask +- **THEN** the mask is cropped without running the model again + +### Requirement: Whole frame silhouette +The user SHALL be able to choose Whole frame as the silhouette source, which makes the whole cropped image the material, so the keychain is a plaque whose shape comes from the base plate options. Controls that only matter for a traced silhouette (threshold, close gap, fill areas, specks) SHALL NOT be shown for it. + +#### Scenario: Plaque +- **WHEN** Whole frame is chosen with a rounded rectangle plate +- **THEN** the keychain is a rounded rectangular plaque carrying the photo relief + +#### Scenario: Edge +- **WHEN** the relief reaches the plaque's edge +- **THEN** it rolls off to the rim height and does not end as a vertical wall + +### Requirement: Detail as engraving +The Detail source SHALL offer three modes: Engrave (the darkest features of the photo are pressed in as grooves and the mid-tones stay smooth), Relief (the tone drives the height) and Both. Engrave SHALL be the default. The user SHALL be able to set how dark a feature must be to be engraved. + +#### Scenario: Engrave +- **WHEN** the mode is Engrave on a face crop +- **THEN** the eyes, the nose and the dark stripes are grooves and the rest of the surface is not bumpy + +#### Scenario: Threshold +- **WHEN** the dark threshold is lowered +- **THEN** more of the photo's darker features are engraved + +#### Scenario: Relief +- **WHEN** the mode is Relief +- **THEN** the height follows the tone as before + +### Requirement: Photo presets +The app SHALL offer presets for the Photo source (Subtle, Portrait, Deep) that set the three shares, the height range, and the detail radius, amount and levels. Each value SHALL stay editable and the panel SHALL then show Custom. + +#### Scenario: Portrait +- **WHEN** the user chooses Portrait +- **THEN** the shares, range and detail values are set to the preset's values + +#### Scenario: Edited +- **WHEN** the user edits any of them afterwards +- **THEN** the preset indicator shows Custom and the value is kept + +### Requirement: Clean depth and printable slopes +The Photo source SHALL snap the depth to the edges of the image so that the subject's depth does not bleed into the background, SHALL limit the steepness of the surface to a maximum slope, and SHALL smooth the result lightly. The user SHALL be able to set the snapping strength, the maximum slope and the final smoothing, and to turn each of them off. With the maximum slope on, no wall of the relief SHALL be steeper than the set angle, except the sides that the silhouette defines. + +#### Scenario: Edge snapping +- **WHEN** the subject has a sharp edge in the image and the depth map is soft there +- **THEN** with snapping on the depth steps where the image edge is, and with snapping off it does not + +#### Scenario: Slope limit +- **WHEN** the mix would make a wall steeper than the maximum slope +- **THEN** the surface is lowered so that the slope between neighbouring points does not exceed the angle + +#### Scenario: Slope off +- **WHEN** the maximum slope is 0 +- **THEN** the surface is not limited + +#### Scenario: Floor and level areas +- **WHEN** the slope limit is applied +- **THEN** no point goes below the minimum height and level areas keep their height + +### Requirement: Subject presets +The app SHALL offer subject presets Face, Whole animal and Drawing / logo, which set the silhouette source, the detail mode and its dark threshold, the part size and the outline for that kind of picture, and SHALL offer to choose one automatically from the picture. The automatic choice SHALL stop as soon as the user edits any of the values, which then show Custom. For Drawing / logo the preset SHALL use the contour only, a 60 mm part, a 1.2 mm bevel, an outline smoothness of 0.15 mm and automatic opening of large areas. + +#### Scenario: Automatic +- **WHEN** an image with a few tones and thin strokes is loaded and automatic choice is on +- **THEN** Drawing / logo is selected + +#### Scenario: Manual +- **WHEN** the user chooses Whole animal +- **THEN** its values are set and automatic choice is turned off + +#### Scenario: Edited +- **WHEN** the user changes any value of a preset +- **THEN** the selector shows Custom and the value is kept + +#### Scenario: Drawing +- **WHEN** Drawing / logo is chosen +- **THEN** the relief uses the contour only and enclosed regions larger than the area threshold are open + +### Requirement: Persistence of the depth result +The depth SHALL be stored with the image in the browser's storage and restored after a reload with the source and controls, without running the model again. Deleting the model from the model manager SHALL NOT remove an existing result; replacing or removing the image SHALL. If the stored result is missing, the app SHALL say so, offer to run again and build the relief without the Depth share meanwhile, never an empty relief. If storage is unavailable the result SHALL work for the session and the user SHALL be warned that it will not persist. + +#### Scenario: Reload +- **WHEN** the user computes the depth, sets the shares and reloads +- **THEN** the same relief is shown without running the model + +#### Scenario: Result lost +- **WHEN** the stored depth is gone but the Depth share is above zero +- **THEN** a message offers to run again and the relief uses the other sources until then + +### Requirement: Availability and hints +The Photo source SHALL be offered only in Image mode and SHALL NOT be offered for SVG images. The app SHALL tell the user when the detail asked for is finer than the grid can carry (a finer quality setting is needed) and when the subject's face is too small in the part to read (crop closer or use Whole frame). Hints SHALL never block an action. + +#### Scenario: Shape mode +- **WHEN** the mode is Shape +- **THEN** the Photo source is not offered + +#### Scenario: Coarse grid +- **WHEN** the grid cell is larger than about a third of the detail radius +- **THEN** a hint suggests a finer quality + +#### Scenario: Small face +- **WHEN** the detail radius is only a few grid cells at the current crop +- **THEN** a hint suggests cropping closer or using Whole frame + +### Requirement: Smooth outline +The outline of a keychain made from an image SHALL be smooth: it SHALL NOT show steps from the pixels of the image, from the cells of the relief grid or from the resolution of the mask, and its position SHALL follow the edge of the subject to a fraction of a grid cell. Sharp corners that exist in the subject, such as the tip of a thin spike, SHALL stay sharp while the curves between them are smoothed. The user SHALL be able to set the smoothness and the angle from which a vertex counts as a corner. + +#### Scenario: No steps +- **WHEN** the silhouette of a photo or a drawing is made into a keychain at any quality setting +- **THEN** the side wall follows a smooth line and shows no staircase + +#### Scenario: Spike tips +- **WHEN** the drawing has a thin pointed spike +- **THEN** the spike keeps its tip and its length within the grid cell + +#### Scenario: Advanced controls +- **WHEN** the user opens the Advanced group of the outline +- **THEN** the corner angle and the sampling step can be set, and their defaults are restored by reset + +#### Scenario: Independent of the grid +- **WHEN** the quality setting is made coarser +- **THEN** the outline stays smooth and only the relief detail becomes coarser + +### Requirement: Compatibility +Keychains saved before this feature SHALL load unchanged. With the Distance or Brightness source, no crop and a traced silhouette, the relief SHALL be identical to what it was before this feature. + +#### Scenario: Old record +- **WHEN** a record without the new fields is loaded +- **THEN** it opens with its former source, the full frame, and produces the same relief as before + +### Requirement: Language +All text of the Photo source, the crop, the whole-frame silhouette, the presets and the hints SHALL be available in every language of the app. + +#### Scenario: Russian +- **WHEN** the interface language is Russian +- **THEN** the source choice, shares, crop controls, presets and hints are in Russian diff --git a/openspec/changes/34-keychain-photo-relief/tasks.md b/openspec/changes/34-keychain-photo-relief/tasks.md new file mode 100644 index 0000000..12de86b --- /dev/null +++ b/openspec/changes/34-keychain-photo-relief/tasks.md @@ -0,0 +1,35 @@ +# Tasks + +## 1. Pick the mix on the owner's photos (spike) + +- [ ] 1.1 With the PoC combiner (`poc/ai/`, "Shape combiner") on the cat photo, the face crop `cat_face`, a human portrait and a plush toy: run a matte model and a depth model on the same image, then choose the shares of Contour, Depth and Detail, the detail radius, amount and levels, and the contour bevel that make the face readable in the shaded relief. Record the chosen values for the Portrait, Subtle and Deep presets in `design.md` (Open Questions) and `poc/ai/FINDINGS.md`; record whether depth on the crop beats depth on the whole frame, and compare Depth Anything 3 Small if an ONNX exists. If no mix makes a face recognisable, stop and update the artifacts with `/opsx:update`. + +## 2. Geometry (React-free) + +- [ ] 2.1 `geometry/raster/tone.ts`: `localContrast(gray, w, h, radiusPx, amount)`, `quantise(values, levels, softness)` and `normaliseDepth(plane, mask)` (2nd–98th percentile inside the mask, clamped 0…1). Tests: amount 0 gives a flat 0.5, a constant plane gives no NaN, a small bump on a gradient survives the boost and the mean stays about 0.5, quantise gives exactly `levels` distinct values at softness 0 and monotone ramps at softness 1, depth normalisation ignores outliers and uses only the mask, a constant depth gives a flat result. +- [ ] 2.2 `SourceParams.kind: 'photo'` with `mix {contour, depth, detail}`, `depthId`, `boost`, `boostRadius`, `levels`, `levelSoftness`; `registerDepth` and `depthRegistry`; `buildGrid` builds `value = (wC·contour + wD·depth + wT·detail) / Σw` through the brightness path (contrast, gamma, invert, smoothing, rim ceiling, clamp); missing depth counts as zero share and is reported in `KeychainInfo`; radius mm → pixels from the crop and part size. Tests: with `distance` or `brightness` the existing regression volumes and bounding boxes are unchanged; Contour only equals the Distance source; shares are proportions (doubling changes nothing); a synthetic dome depth gives a dome relief inside the mask and the edge still rolls off; a far background does not flatten a shallow subject; Detail does not change the mean height; the same radius in mm gives the same detail scale at two resolutions; missing depth reports and uses the other shares; sum zero falls back to Contour. +- [ ] 2.3 `KeychainParams.crop` and `silhouette.source: 'frame'` (full mask): the cropped view of the planes (gray, alpha, matte, depth) in `buildSilhouette` and `buildGrid`. Tests: no crop leaves volumes unchanged; a half-width crop halves the part box and keeps the relief of that region; the frame source makes the whole crop material and hides tracing steps; the matte is cropped without re-registration. +- [ ] 2.3a Exact outline for image silhouettes: the sub-pixel outline from the continuous field (`traceAlpha`), simplification and corner-preserving smoothing, the relief clipped by the polygon as `CrossSection` (as `shapeCrossSection` does in Shape mode), large areas open and small ones filled by the existing area rules. The outline smoothing keeps sharp corners: resample, find corners by the turning angle, Gaussian along the curve with corners fixed. Tests: the outline of a synthetic disc and of a thin spike deviates from the true shape by less than 0.1 grid cell at two qualities, no vertex of the wall lies on a grid line pattern (no staircase: the number of direction changes of the outline stays close to the true shape's), a spike tip keeps its length within one cell, a coarser quality changes only the relief and not the outline, bounding box and volume regressions of the existing keychain tests stay within their tolerances for a traced silhouette. +- [ ] 2.3b Detail modes Engrave / Relief / Both with the dark threshold (`1 − groove`), default Engrave; presets Face, Whole animal and Drawing / logo (chosen automatically for few-tone thin-stroke images). Tests: Engrave leaves mid-tones at one level and lowers only pixels darker than the threshold, a lower threshold engraves a superset, Relief equals the former tone mapping, the Drawing preset leaves a one-tone image with contour only. +- [ ] 2.3c Clean depth and slopes in `geometry/raster/`: `guidedFilter(depth, guide, w, h, radiusPx, eps)`, `limitSlope(height, w, h, cell, maxSlopeDeg, fixed, floor)` (forward and backward sweeps) and their use in `buildGrid` in the order of the design (`snap`, `maxSlope`, `finalSmooth` params with defaults 60, 50°, 0.2 mm). Tests: the guided filter keeps a step that coincides with a guide edge and flattens noise where the guide is flat, strength 0 returns the input, a constant plane stays constant; the slope limit keeps every neighbour difference within the bound on a spike and on a cliff, never raises a node, never goes below the floor, leaves `fixed` nodes alone, angle 0 is the identity; the existing source regression volumes are unchanged; the result does not depend on the image resolution at the same mm values. +- [ ] 2.3d Subject presets Face, Whole animal and Drawing / logo as data, `presetOf` with Custom, and the automatic chooser (`chooseSubject(stats, mask box)`: tone count, stroke thinness, box fill). Tests: each preset sets exactly its listed values, a few-tone thin-stroke fixture selects Drawing, a frame-filling subject selects Whole animal, a face crop selects Face, editing any value gives Custom and turns the automatic choice off, defaults of an old record are unchanged. +- [ ] 2.4 `migrateKeychain` fills the new fields for old records and normalises partial values. Tests: round trip, partial and corrupt values fall back to defaults, old records keep their source. + +## 3. ML and image side + +- [ ] 3.1 `src/ml/` registry gets `kind: 'depth'` and external-data support (path, bytes and SHA-256 of the data file passed to the runtime as external data); the manager shows the kind and the picker filters by it. Tests: registry validation of the data file, download of both files with progress and cancel, a missing or mismatched data file fails the model, depth models are listed separately. +- [ ] 3.2 `features/keychain/depth.ts` and `useDepth`: run through the client on the crop (aspect kept, multiple of 14, inference cap 2048, resample back, normalise), store in `imageStore` as a `depth` record keyed by image, model, revision and crop, restore on load, mark "for another crop" when the crop differs, register in the worker. Tests with a mocked client: input geometry for wide and tall crops, output mapped back, cancel and latest-wins, restore does not call the client, a changed crop keeps the record and marks it, replacing the image drops it, deleting the model keeps it. + +## 4. UI + +- [ ] 4.1 `CropEditor` (drag, handles, aspect lock, fit to subject, reset), live update of the mask preview and relief, also reachable from the enlarged mask dialog. Strings in `en.json` and `ru.json`. No artboard exists: built from `ui/` components and `theme.css` tokens, said in the commit; compare in en and ru and at 390 px. Component tests: dragging and handles change the crop, aspect lock, fit to subject uses the mask box, reset, keyboard nudging, minimum size, key parity. +- [ ] 4.2 Relief source Photo and `PhotoReliefBlock`: three share sliders with a stacked bar, depth model picker and Compute depth (disabled with a hint and a manager link without a depth model, `AiStatus` with loader, current line and Cancel, "for another crop" note, missing-depth message), Detail boost (amount, radius as `LengthField`), Tone levels (count, softness), presets Subtle / Portrait / Deep with Custom, the Subject selector (Face, Whole animal, Drawing / logo) with Choose automatically, Snap depth to edges, Max slope (degrees) and Final smoothing (`LengthField`), `OutlineControls` in the Image section (smoothness, collapsed Advanced group with corner angle and sampling step), existing contrast, gamma, invert and smoothing, height greyscale view, hints (coarse grid, small face); silhouette source "Whole frame" hiding the tracing controls. Component tests: availability per mode and for SVG, shares update the relief with no client call, no depth model disables the button with a hint and a manager link, and it becomes active when the model store reports one; the loader and the current line follow the events of a fake run, presets, subject presets and Custom, the automatic choice stops on an edit, the Advanced group holds corner angle and step with reset, snapping, slope and smoothing controls update the relief with no client call, lost depth falls back with a message, hints, key parity, steppers on every numeric input. + +## 5. Version + +- [ ] 5.1 Bump to 1.6.0: `package.json` and a `CHANGELOG.md` entry `1.6.0 — 34-keychain-photo-relief`. Test: the existing version test passes with the new number. + +## 6. Verification + +- [ ] 6.1 `npm test` and `npm run build` are green, the dev server starts, no console errors; with the Distance and Brightness sources the existing keychain tests and volumes are unchanged. +- [ ] 6.2 In the browser, en and ru and at 390 px, with the cat photo and a portrait: AI silhouette, crop to the face, Compute depth, the Portrait preset, the eyes, nose and ears are visible in the 3D relief, the shares update live without a model run (network panel), Whole frame gives a plaque, reload restores crop, shares and depth without a run, deleting the model keeps the relief, the STL is one valid piece that prints face up, no request carries image data. diff --git a/openspec/changes/35-smooth-outline/.openspec.yaml b/openspec/changes/35-smooth-outline/.openspec.yaml new file mode 100644 index 0000000..7cc41bb --- /dev/null +++ b/openspec/changes/35-smooth-outline/.openspec.yaml @@ -0,0 +1,2 @@ +schema: spec-driven +created: 2026-10-22 diff --git a/openspec/changes/35-smooth-outline/design.md b/openspec/changes/35-smooth-outline/design.md new file mode 100644 index 0000000..802395b --- /dev/null +++ b/openspec/changes/35-smooth-outline/design.md @@ -0,0 +1,25 @@ +# Design + +## Context + +`traceSilhouette(seg, id, paper)` blurs the label mask (σ = 1 px at 4 px/mm), traces the 128 iso-line (`traceAlpha`), keeps the largest outer loop and simplifies it (`simplifyLoop`, 0.15 mm). The polygon becomes an `OutlineSource` cutout; `buildCutout` grows it by the clearance with `CrossSection.offset` and extrudes it. The result is a faithful but noisy polygon of about 3000 vertices per tool. + +## Decisions + +- **Fit curves, then sample them.** Schneider's algorithm (the one used by Potrace-like and vector-editor tools): chord-length parametrisation, least-squares cubic with given end tangents, Newton–Raphson reparametrisation, split at the point of the largest error until the error is below the tolerance. Runs between corners; between runs the tangent is not shared (a real corner), inside a run the split point's tangent is shared (G1). A loop without corners is cut in two with central-difference tangents. Tolerance default **0.25 mm** and sampling step **1 mm**, the values the owner judged best in the PoC (0.15 mm and 0.5 mm kept a faint ripple on a screwdriver neck; 0.25 and 1 gave the smoothest walls); the previous `OUTLINE_TOLERANCE` of 0.15 mm applies only to the "original polyline" option. +- **Noise first, then fit.** The loop is resampled uniformly (≈ 0.17 mm), smoothed along the curve with corners held fixed (Gaussian σ default 0.25 mm: the one tuned for cutouts), then fitted; the fit tolerance absorbs what is left of the noise. The corner detector (turning angle over a ±3-vertex window, local maxima, default 50°) keeps pointed tips sharp. +- **Sampling.** Each cubic is sampled at about 1 mm along its length (uniform in `t`); the chord error is `step² / (8 R)`, below 0.06 mm for curvature radii above 2 mm, inside the 0.25 mm fit tolerance. Where the radius is smaller the sampling is refined (step ≤ `sqrt(8 R · 0.03)`). The sampled polygon is what goes to the geometry kernel (Manifold needs polygons) and to STL; the curve is an intermediate representation. +- **Offset on the curve.** The clearance is applied to the samples along the analytic normals; at a convex corner (left turn of a counter-clockwise loop) round-join points at 15° steps, at a concave corner a mitre on the bisector (scale limited). Where this would self-intersect (clearance larger than the local radius) the kernel's offset (`CrossSection.offset`) is the fallback, as today. +- **Same stage for every source.** The stage takes a label mask or a soft field, so classic, AI (with the bicubic soft mask and the optional second pass of `32`) and keychain masks all go through it; the baseline detection code is untouched. +- **Advanced group.** The user-facing controls are Outline smoothness and Fit tolerance; the **corner angle** (degrees, default 50) and the **sampling step** (`LengthField`, default 1 mm) are rarely needed and sit in a collapsed "Advanced" group under them, with a reset to the defaults. `OutlineControls` (introduced for Keychain in `34`) is the one component, extended here with the tolerance and the polyline option and reused by the photo dialog; a lower step or angle gives a finer outline, a higher one fewer vertices (the vertex count is shown). Values are stored with the cutout session or keychain params and migrated with defaults. +- **Opt-out.** "Use the original polyline" skips the fit and smoothing, reproducing the previous polygon exactly (this is also what the regression tests compare against). +- **Stored data.** An outline cutout keeps `{points, size}`; the points are now the sampled fitted polygon. Old sessions have denser polygons and load unchanged. Optionally the control points are stored with the points for later editing (not used yet). +- **Keychain.** The same stage gives the outline of an image silhouette in Keychain: in the PoC the drawing `check` outline (3475 polyline vertices) became 40 Bézier segments (121 control points) and 262 vertices in the model, with wall normals taken analytically from the curve, so the wall shows no ripple either. The top surface of the keychain stays a height field cut along the fitted outline. +- **Evidence (PoC).** Owner's flash photo, A4: classic outlines 8902 polyline vertices → 248 Béziers / 747 control points / 3214 sampled vertices after offset with round joins; AI 304 Béziers. Walls without ripple at a grazing view; the previous polyline with window-averaged normals still showed ripple. + +## Risks / Trade-offs + +- [Fitting rounds a real small feature] → tolerance 0.15 mm and corner detection; the control is exposed; the polyline option remains. +- [A spike thinner than the smoothing disappears] → corner-preserving smoothing and a smaller σ; checked by a test on a thin spike. +- [Self-intersection of the offset at tight concave features] → fallback to the kernel's offset. +- [Fit time on large outlines] → the fit is O(n) per iteration over a few thousand points per loop, milliseconds; done once per silhouette and cached with the outline. diff --git a/openspec/changes/35-smooth-outline/proposal.md b/openspec/changes/35-smooth-outline/proposal.md new file mode 100644 index 0000000..d85a489 --- /dev/null +++ b/openspec/changes/35-smooth-outline/proposal.md @@ -0,0 +1,29 @@ +# Proposal + +## Why + +A pocket in Cutout mode (and the outline of a keychain made from an image) is today a polygon of thousands of vertices traced from a pixel mask (`traceSilhouette`: blur, iso-line, simplification by 0.15 mm). The polygon follows every bit of noise of the detected edge, so the printed wall shows a fine vertical ripple, and the model carries thousands of needless vertices. The photo contour is also not "ideal" to begin with: what the user wants is the **shape** of the tool, not every wobble of the detection. + +In the PoC (`poc/ai/`) a pocket built from fitted curves instead of the raw polyline has no ripple, and the same fit works for the classic and the AI silhouettes: on the owner's photo the classic outlines (8902 polyline vertices) become 248 cubic Bézier segments (747 control points), the AI outlines 304. + +## What Changes + +- **One shared outline stage for every source of silhouette.** After the silhouette is found (classic detection, AI mask, or a keychain mask), the outline is traced, smoothed along the curve (sharp corners kept) and **fitted with cubic Bézier curves** (Schneider's algorithm) to a tolerance (default 0.25 mm), corners detected by the turning angle (default 50°) staying real corners with G1-continuous curves in between. +- **Compact and smooth geometry.** The fitted curves are sampled into a polygon with a chord error well below the tolerance (a step of about 1 mm along the curve), so the model gets a few hundred vertices instead of thousands and walls without ripple. Clearance offsets are made on the curve (round joins at convex corners, mitre at concave ones). +- **Applies to both engines.** The classic detection (the baseline) is not changed: only the stage after it, so the baseline silhouettes stay identical and its outline gets smoother. The AI engine of `32` uses the same stage. +- **Controls.** "Outline smoothness" (σ, default 0.25 mm) and "Fit tolerance" (default 0.25 mm) and "Sampling step" (default 1 mm) under the silhouette step, with the vertex count shown; "Use the original polyline" as an option that restores the previous behaviour. +- **Keychain.** The image and shape silhouettes of Keychain use the same stage for their outline (see `34`). + +## Capabilities + +### New Capabilities +- `smooth-outline`: fitting of smooth curves with few points to silhouette outlines, shared by Cutout (both engines) and Keychain. + +### Modified Capabilities + +## Impact + +- New pure module `geometry/outlineFit.ts` (resample, corner detection, along-curve smoothing, Schneider fitting, sampling, offset), used by `traceSilhouette` callers in `features/cutout/photo/createCutouts.ts` and by the keychain outline; `segment.ts`, `colour.ts` and the classic detection are not edited, their frozen-output test (`32`, task 2.1) stays green; the `OutlineSource` points of a cutout become the sampled fitted polygon (the stored `points` keep their shape, so saved sessions load unchanged); strings in en and ru, tests. +- Out of scope: changing the classic silhouette detection, editing the outline by hand (node dragging), exporting curves as such (STL and 3MF stay polygonal), smoothing of mesh cutouts (STL imports). +- Version 1.7.0 (minor: a new option that also improves existing output). +- Depends on: `25a-photo-trace-improvements`, `29-beta-label`, `34-keychain-photo-relief` (the Keychain outline and `OutlineControls` start there; this change generalises them); `32` uses it but is not required. diff --git a/openspec/changes/35-smooth-outline/specs/smooth-outline/spec.md b/openspec/changes/35-smooth-outline/specs/smooth-outline/spec.md new file mode 100644 index 0000000..54cc124 --- /dev/null +++ b/openspec/changes/35-smooth-outline/specs/smooth-outline/spec.md @@ -0,0 +1,77 @@ +# Spec Delta + +## Purpose +Defines the shared stage that turns a detected silhouette into a smooth outline with few points, used by Cutout with either detection engine and by Keychain. + +## ADDED Requirements + +### Requirement: Smooth outline from any silhouette +The outline made from a silhouette, whether found by the classic detection, by the AI engine or from a keychain image or shape, SHALL be smooth: it SHALL be fitted with smooth curves instead of following every pixel and every wobble of the detected edge, and SHALL be turned into a polygon with a small number of vertices. Sharp corners of the subject, such as the tip of a tool, SHALL stay sharp, and the curves between corners SHALL be continuous in direction. The user SHALL be able to set the smoothness and the fit tolerance and to choose to keep the original polyline. + +#### Scenario: Smooth walls +- **WHEN** a pocket is made from a detected tool and viewed along its wall +- **THEN** the wall shows no ripple that follows the pixels or the noise of the detected edge + +#### Scenario: Fewer points +- **WHEN** the outline of a tool is made with the default tolerance +- **THEN** its polygon has at least five times fewer vertices than the original polyline, and no point of it is farther than the tolerance from the original polyline + +#### Scenario: Sharp tips +- **WHEN** a tool has a pointed tip +- **THEN** the tip stays pointed and its position is within the tolerance of the original + +#### Scenario: Both engines +- **WHEN** the same photo is processed by the classic and the AI engine +- **THEN** both outlines go through the same stage and both come out smooth + +#### Scenario: Original polyline +- **WHEN** the user chooses to keep the original polyline +- **THEN** the outline is the same as before this feature + +### Requirement: Advanced outline settings +Besides the smoothness and the fit tolerance, the user SHALL be able to set the angle from which a vertex counts as a corner and the sampling step of the polygon, in an Advanced group that is collapsed by default and can be reset to the defaults. The same controls SHALL be offered for Cutout and for Keychain. + +#### Scenario: Collapsed +- **WHEN** the outline controls are first shown +- **THEN** smoothness and tolerance are visible and the corner angle and sampling step are in a collapsed Advanced group + +#### Scenario: Finer step +- **WHEN** the sampling step is made smaller +- **THEN** the polygon has more vertices and follows the curve more closely + +#### Scenario: Corner angle +- **WHEN** the corner angle is raised above the turning angle of a tip +- **THEN** that tip is smoothed instead of kept sharp + +#### Scenario: Reset +- **WHEN** the user resets the Advanced group +- **THEN** the angle returns to 50 degrees and the step to 1 mm + +### Requirement: Clearance on the smooth outline +The clearance of a cutout SHALL be applied to the smooth outline so that the pocket stays smooth: rounded at convex corners, mitred at concave ones, and never smaller than the outline plus the clearance. + +#### Scenario: Convex corner +- **WHEN** the clearance is 0.8 mm and the outline has a convex tip +- **THEN** the pocket around the tip is rounded with a radius of the clearance + +#### Scenario: Containment +- **WHEN** a pocket is made with a clearance +- **THEN** the tool's outline lies inside the pocket everywhere + +### Requirement: Detection unchanged +Adding the smooth outline SHALL NOT change which silhouettes the classic detection finds, their sizes or their numbering; only the polygon made from each of them. + +#### Scenario: Same silhouettes +- **WHEN** the reference photos are processed +- **THEN** the number, sizes and areas of the silhouettes are identical to those before this feature + +### Requirement: Compatibility and language +Sessions saved before this feature SHALL load unchanged. All text of this feature SHALL be available in every language of the app. + +#### Scenario: Old session +- **WHEN** a session with outline cutouts saved before this feature is loaded +- **THEN** the cutouts are as they were + +#### Scenario: Russian +- **WHEN** the interface language is Russian +- **THEN** the smoothness, tolerance and polyline option are in Russian diff --git a/openspec/changes/35-smooth-outline/tasks.md b/openspec/changes/35-smooth-outline/tasks.md new file mode 100644 index 0000000..640a555 --- /dev/null +++ b/openspec/changes/35-smooth-outline/tasks.md @@ -0,0 +1,21 @@ +# Tasks + +## 1. Geometry (React-free) + +- [ ] 1.1 `geometry/outlineFit.ts`: `resample`, `findCorners`, `smoothLoop` (Gaussian along the loop with corners fixed), `fitLoop` (Schneider: chord-length parametrisation, least squares, reparametrisation, split) and `sampleCurves`. Tests: the fit of a circle uses few segments and stays within the tolerance, of a square keeps four corners, of a noisy circle absorbs the noise within the tolerance, a thin spike keeps its tip within the tolerance, a loop without corners is closed with continuous direction, the number of vertices drops at least five times on a traced tool outline fixture and the maximum distance to the original stays below the tolerance. +- [ ] 1.2 `offsetCurve(samples, d)`: the clearance along the analytic normals, round join at convex corners, mitre at concave ones, fallback to the kernel's offset when the result self-intersects. Tests: the offset of a circle is a circle of radius + d, a convex tip is rounded with radius d, the tool's outline lies inside the offset everywhere, a tight concave notch falls back without a crash. +- [ ] 1.3 Use in `createCutouts` (`tracedCutouts`) with `traceSilhouette` kept as it is: the traced loop goes through the fit, `OutlineSource.points` is the sampled polygon, an option keeps the original polyline. Tests: with the polyline option the existing outline tests are unchanged; with the fit the vertex count drops and the polygon stays within the tolerance of the original; classic silhouettes of both fixtures are identical to the frozen baseline (`32` task 2.1); the AI engine of `32` goes through the same path. +- [ ] 1.4 Keychain outline (silhouette and shape plates) through the same stage as in `34`. Tests: the existing keychain regression volumes stay within their tolerances; the outline has fewer vertices and no ripple. + +## 2. UI + +- [ ] 2.1 Extend `OutlineControls` (introduced for Keychain in `34`) and use it under the silhouette step of the photo dialog: outline smoothness and fit tolerance (lengths, so `LengthField`), the vertex count, the option "original polyline", and a collapsed Advanced group with corner angle (degrees) and sampling step (`LengthField`) and a reset; strings in `en.json` and `ru.json`. No artboard exists: built from `ui/` components and `theme.css` tokens, said in the commit; compare in en and ru and at 390 px. Component tests: controls update the outline and the count, the Advanced group is collapsed by default and its reset restores 50° and 1 mm, a finer step raises the vertex count, the polyline option restores the previous result, steppers on every numeric input, key parity. + +## 3. Version + +- [ ] 3.1 Bump to 1.7.0: `package.json` and a `CHANGELOG.md` entry `1.7.0 — 35-smooth-outline`. Test: the existing version test passes with the new number. + +## 4. Verification + +- [ ] 4.1 `npm test` and `npm run build` are green, no console errors; the classic detection output on both fixtures is unchanged. +- [ ] 4.2 In the browser, en and ru and at 390 px, with the owner's two photos and both engines: pockets show smooth walls at a grazing view in the 3D preview, the vertex count is several times lower, tips stay pointed, the clearance gives rounded convex corners, the exported STL of the bin is one valid solid, saved old sessions load unchanged. diff --git a/openspec/changes/36-site-refresh/.openspec.yaml b/openspec/changes/36-site-refresh/.openspec.yaml new file mode 100644 index 0000000..7cc41bb --- /dev/null +++ b/openspec/changes/36-site-refresh/.openspec.yaml @@ -0,0 +1,2 @@ +schema: spec-driven +created: 2026-10-22 diff --git a/openspec/changes/36-site-refresh/design.md b/openspec/changes/36-site-refresh/design.md new file mode 100644 index 0000000..4fc352f --- /dev/null +++ b/openspec/changes/36-site-refresh/design.md @@ -0,0 +1,52 @@ +# Design + +## Context + +Card texts are i18n keys: `products..{name,desc,tag1,tag2}` for the root page and `start..{desc,tag1,tag2}` for the mode pages, with a parity test between `en.json` and `ru.json`. Illustrations are SVG files rendered by `scripts/render-art.ts` from `generate()` meshes in groups with their own scale (Gridfinity three with one scale, Keychain two, each Rack part alone); the Keychain card is aligned to the box by `BOX_ART_HEIGHT`. `ModeArt.test.tsx` checks the 60 KB budget. + +## Goals / Non-Goals + +**Goals:** +- Texts that say what each card is, in the same style, without placeholders or jargon, in both languages. +- A README that matches the product. +- Illustrations that sit on the same visual line, with the Gridfinity cards as the reference. + +**Non-Goals:** +- New copywriting for workspaces, screenshots, marketing pages. + +## Decisions + +- **Card text rules.** A description is one or two sentences: what you get, then the one thing that makes it useful; it names the input when there is a special one (a photo, an image, a device). Two tags per card, each at most two short words, nouns or noun phrases, no arrows except "image → relief" and "device → shell" where the arrow is the point; no status words (soon, new, beta). Russian: translated, sentence case, no Latin words except names; "keystone" is written as a name (`keystone`) in the same form on every card. +- **Proposed texts (to be confirmed by the owner in the first task).** + + | Card | EN description | EN tags | RU description | RU tags | + |---|---|---|---|---| + | Gridfinity Master | Parametric Gridfinity bins, baseplates and tool cutouts, with pockets traced from a photo. | bins · baseplates; tool cutouts | Параметрические контейнеры и плиты Gridfinity, вырезы под инструменты, в том числе по фото. | контейнеры · плиты; вырезы под инструмент | + | Keychain Master | Turn a picture or a plain shape into a relief keychain: silhouette, height pattern, text, ring mount, two halves with pins. | image → relief; shapes · text | Брелок-рельеф из картинки или простой фигуры: силуэт, узор высоты, надпись, крепление под кольцо, две половинки со штифтами. | картинка → рельеф; фигуры · надпись | + | Rack Master | Parts for a 10-inch rack: panels with patterns, enclosures around a device and shelves, with standard ear holes. | 1–10 U; panels · cases · shelves | Детали для 10-дюймовой стойки: панели с узорами, корпуса под устройство и полки со стандартными отверстиями в ушах. | 1–10 U; панели · корпуса · полки | + | Cutout | A bin with pockets shaped to your tools: from a photo on a sheet of paper, an STL, a circle or a rectangle. | from photo; STL import | Контейнер с гнёздами под ваши инструменты: по фото на листе бумаги, из STL, круг или прямоугольник. | по фото; импорт STL | + | Keychain Image | A relief keychain from a picture: a drawing, logo or emblem becomes a silhouette with a height pattern, text and a ring mount. | image → relief; text · mounts | Брелок-рельеф по картинке: рисунок, логотип или эмблема становятся силуэтом с узором высоты, надписью и креплением под кольцо. | картинка → рельеф; надпись · крепление | + | Rack Enclosure | A front plate with rack ears and a shell around your device: mini PC, switch or router. Window, `keystone` sockets, holes and vents. | device → shell; keystone sockets | Передняя панель с ушами и оболочка вокруг устройства: мини-ПК, коммутатор или роутер. Окно, гнёзда `keystone`, отверстия и вентиляция. | устройство → корпус; гнёзда keystone | + + Box, Baseplate, Keychain Shape, Rack Panel and Rack Shelf keep their descriptions and get tags in the same style ("grid × height", "compartments"; "fractional cells", "split to the bed"; "shapes", "rim"; "1–10 U", "patterns"; "plate + shelf", "ventilated"). The final wording is settled in task 1.1. +- **README structure.** Title and one-line pitch (unchanged); a status line (beta, current version, link to the changelog); Why this exists (the motivation text, identical to the site); What you get: the product table with every mode and its main options, the highlights; Privacy: the work stays in the browser, photos and files are never uploaded, visit counting only after consent and the repository link to its settings; Quick start and commands; Deploy; How it works; Project layout (listing the folders that exist, such as `poc/`); Roadmap and changes (`openspec/ROADMAP.md`, `CHANGELOG.md`); References; License. The statement "No server. No account." is kept for the app and followed by the privacy note. A test (or the existing `site.test.ts`) checks that the motivation text in the README equals the site's. +- **Keychain art from the ring emblem.** `scripts/art/ring.jpg` is the PoC picture; `npm run art` decodes it with `jpeg-js` (already a dev dependency), makes a `GrayImage`, runs the Keychain Image pipeline (`generate('keychain')`) with default relief and size about 50 mm, the large enclosed area opened (the emblem's hole) and the small ones between strokes filled (the same rule as the areas of change 18), and draws it like the other keychain art, with the existing smoothing. No eyelet is added: the emblem already has a through hole, and an eyelet on a spiky silhouette looks accidental; the alternative is a small eyelet at the top spike, rejected for the look. The paw module `geometry/art/paw.ts` stays for tests until it is no longer used, then is removed. +- **Scale and air.** The Gridfinity cards are the reference: their drawing keeps the margin that the owner is happy with. The Keychain Image card is scaled so its drawn height is a fixed fraction of the box drawing's (`KEYCHAIN_ART_RATIO`, about 0.8, set by eye against the cards in the browser and written down) instead of equal to it, and the width limit stays. Every illustration is measured by `projectedSize` in the script and written to `src/app/art/manifest.json` as fractions of the 250 × 165 field; the test asserts at least 6 % of free space on each side for every card and that the Keychain Image drawing is not taller than the Gridfinity box. +- **Rack at 1 U.** The panel is `defaultRackPanel` with `units: 1` and the Skadis pattern (the 1 U plate has less room, so the pattern's region is checked: a plate that shows no pattern element is not acceptable, the pattern pitch may be reduced for the art only); the enclosure uses a small device (about 25 mm high so that the shell fits 1 U) with one keystone; the shelf uses `units: 1` with the default shelf and side walls capped to what a 1 U plate allows. The three Rack drawings share one scale chosen so the widest of them (the 254 mm width) fits the field with the Gridfinity margin; they are drawn in one group of the script, which also makes their sizes comparable. +- **Hue per product.** `Product.hue: 'gridfinity' | 'keychain' | 'rack'` (the id doubles as the hue name). `theme.css` keeps `--color-art-0…6` for Gridfinity (blue, unchanged) and adds `--color-art-keychain-0…6` (green) and `--color-art-rack-0…6` (yellow), with the same lightness steps as the blue ladder so shading reads the same; the edge colour is shared. `ModeArt` gets the hue from the art id (`box`, `baseplate`, `cutout` → Gridfinity; `keychainImage`, `keychainShape` → Keychain; `rackPanel`, `rackCase`, `rackShelf` → Rack) and sets `data-hue` on its wrapper; the rules `[data-hue=keychain] .art-t0 { fill: var(--color-art-keychain-0) }` and so on override the fills, so the SVG files are untouched and the card art still follows the tokens. Starting values (to tune by eye against the dark surface): green `#1b2a22 #25402f #345a43 #437557 #58926d #78b08a #a1cfb0`, yellow `#2a2814 #45401a #625b21 #827927 #a39a33 #c5bb52 #e3dc86`. Yellow is kept warm-yellow, not orange, so it does not read as the Okabe–Ito orange of the cutout colours. +- **Logo.** `Logo` takes `hue` and draws its faces from tokens `--color-logo-{top,left,right}` per hue (blue `#7c9cff / #4a63b8 / #34478f` as now; green about `#7edc9d / #3f9a5e / #2e7547`; yellow about `#f0d95c / #b89e2a / #8a7620`), the dark outline unchanged. `Header` passes the active product's hue; on the root page and on routes without a product the logo is blue. The product name text, the buttons and everything else in the header keep their colours. +- **Not coloured.** The interface accent (`--color-accent`), focus rings, selected states, banners and the 3D viewport keep their colours in all products; only the pictures and the logo are coded by product. The colour is never the only cue: the product name and icon stay. +- **Weight and tokens.** The 60 KB budget and the colour tokens stay; if a 1 U drawing is heavier, the smoothing or the trace tolerance is raised for it. + +## Risks / Trade-offs + +- [The ring is a drawing, so the render is a flat plaque with the emblem's silhouette] → the relief profile (bevel) and the two tone passes give it volume; if it looks flat the default profile "dome" is used for the art only. +- [A 1 U panel is a very thin strip and looks small on the card] → the shared Rack scale and the isometric view keep it readable; the card height does not change; the check is by eye in the browser. +- [Texts drift again] → the style rules are in the spec; a test keeps the tags to two per card and at most two words each, and forbids the words "soon" and "скоро" on cards. +- [Green and yellow art looks muddy on the dark card] → the ladders keep the lightness steps of the blue one, values are tuned by eye in the browser; the 6 % margin and weight checks do not depend on colour. +- [The README repeats the specs] → it stays a summary with links, not a copy. + +## Open Questions + +- The exact wording of the cards and the README is the owner's call; the table above is the proposal. +- Whether the ring eyelet question should be revisited after seeing the render. diff --git a/openspec/changes/36-site-refresh/proposal.md b/openspec/changes/36-site-refresh/proposal.md new file mode 100644 index 0000000..c534a66 --- /dev/null +++ b/openspec/changes/36-site-refresh/proposal.md @@ -0,0 +1,34 @@ +# Proposal + +## Why + +The site and the README were written when PrintKit had three Gridfinity modes and a panel. Since then it has gained Cutout from a photo and a circle, Keychain Image and Shape with text, areas and two halves, Rack Enclosure, Shelf and the Kumiko patterns, a beta label and a versioning rule. The texts did not follow: + +- the Keychain product card still carries the tag "soon", the Rack product card describes panels only, the Cutout card tag "snap" and the Keychain Image tag "mask" mean nothing to a newcomer, and the Russian cards mix Latin terms ("keystone"); +- the README's product table misses most of the modes and options, says nothing about the beta status, the version, the changelog or the analytics banner, and "No server. No account." needs the honest footnote that visit counting is optional and asks consent; +- the card illustrations are uneven: the Keychain Image card shows a paw that fills the whole field while the Gridfinity cards leave air around the part, and all three Rack cards show a 2 U part although the common case, and the default of the shelf, is 1 U. + +## What Changes + +- **Texts of the start pages (en and ru).** Rewrite the descriptions and tags of the three product cards and of every mode card so that each states what the part is and what is special about it, with no placeholders ("soon") and no private jargon ("snap", "mask"); two short tags per card, in the same style everywhere; Russian terms translated, a Latin term kept only when it is the name of a thing (Gridfinity, Skadis, keystone) and then in the same form on every card. The tagline, the title and the motivation text are checked against the current products. +- **README.** Bring it up to date: what is in each product and mode, the beta status and where the version and the changelog are, a short privacy note (the work stays in the browser; visit counting only after consent), the commands, the layout including folders that exist (for example `poc/`), the link to the roadmap, the references, and the badges. The motivation text stays identical to the one on the site. +- **Keychain card illustration.** Replace the paw by the ring emblem from the PoC (`poc/ai/samples/ring.jpeg`, an outlined emblem with a large through hole and sharp spikes), rendered by `npm run art` from that picture through the real Keychain Image pipeline, and reduce its scale so that it leaves the same air as the Gridfinity illustrations (which are the reference) instead of filling the field. +- **Rack card illustrations.** All three at 1 U: a 1 U Skadis panel, a 1 U enclosure around a small device, a 1 U shelf; one scale for the three Rack cards (the rack is the same width in all of them) with margins like the Gridfinity cards. +- **Colour by product, in the pictures and the logo only.** The start-page illustrations and the logo in the header take the hue of their product: Gridfinity stays blue (as now), Keychain green, Rack yellow. Only the hue of the illustrations' tone ladder and of the logo's faces changes; the interface (accent, buttons, borders, focus rings, text) keeps its colours in every product. The illustration files do not change (they carry only classes); the hue comes from token sets chosen by the product. On the root page and the start pages the logo stays blue as today. +- **A check for the margins.** `npm run art` records the drawn size of every illustration as a fraction of the card field in a small manifest, and a test keeps every illustration inside a margin and the Keychain illustration no larger than the Gridfinity box. + +## Capabilities + +### New Capabilities + +### Modified Capabilities +- `app-shell`: the card texts and the start page illustrations. +- `site-identity`: the README must be accurate and consistent with the site. + +## Impact + +- `src/i18n/en.json` and `ru.json` (the `start.*` and `products.*` keys of the cards), `README.md`, `scripts/render-art.ts` (the ring source, the Rack models at 1 U, the scales), `scripts/art/ring.jpg` (the source picture), `src/app/art/{keychainImage,rackPanel,rackCase,rackShelf}.svg`, a new `src/app/art/manifest.json`, `ModeArt.test.tsx` and a margin test; `src/app/theme.css` (two more tone ladders and the logo colours per hue), `src/app/ModeArt.tsx` and `Logo.tsx` (a `hue` prop), `Header.tsx` (passes the product's hue); `CLAUDE.md` (the art paragraph). +- No change to geometry, modes or routes. +- Out of scope: new features, screenshots in the README, the product names, the footer and consent texts, texts of the workspaces (panels, sections), texts of the AI changes (they add their own when they land), redrawing the Gridfinity or Keychain Shape illustrations. +- Version 1.7.1 (patch: content and illustrations, no new capability). +- Depends on: `29-beta-label` (the versioning rule); the art uses geometry that is already in the app. diff --git a/openspec/changes/36-site-refresh/specs/app-shell/spec.md b/openspec/changes/36-site-refresh/specs/app-shell/spec.md new file mode 100644 index 0000000..0b50781 --- /dev/null +++ b/openspec/changes/36-site-refresh/specs/app-shell/spec.md @@ -0,0 +1,103 @@ +# Spec Delta + +## MODIFIED Requirements + +### Requirement: Start page illustrations +Each mode card on the Gridfinity, Keychain and Rack start pages and each product card on the root start page SHALL show an isometric illustration that is a render of an actual model produced by the app (a small part with default settings), not a schematic drawing or an icon. The Gridfinity product card SHALL show the box; the Keychain product card SHALL show an image-mode keychain; the Keychain Image and Shape cards SHALL each show a render of their own mode. The image-mode keychain illustration SHALL show a keychain made from a drawing with a complex outline (the ring emblem with sharp spikes and a large through hole), not a circle or other plain geometric figure. The Rack illustrations SHALL show 1 U parts. Illustrations SHALL be static vector images, SHALL follow the app's colour tokens, and SHALL NOT require WebGL, geometry computation or network requests on the start page. + +#### Scenario: Renders match the editor +- **WHEN** the Baseplate card is shown +- **THEN** its illustration is a 2×2 baseplate with the standard cell pockets and the default corner fastener holes, as the editor produces by default + +#### Scenario: Product cards +- **WHEN** the root start page is shown +- **THEN** the Gridfinity card shows the box illustration and the Keychain card shows the image-mode keychain illustration + +#### Scenario: Keychain mode cards +- **WHEN** the Keychain start page is shown +- **THEN** the Image card shows a relief keychain made from the ring emblem, with its through hole open and its spikes sharp, and the Shape card shows a keychain with a geometric outline and rim, each a render of what that mode produces + +#### Scenario: Rack cards at 1 U +- **WHEN** the Rack start page is shown +- **THEN** the Panel, Enclosure and Shelf illustrations each show a part one rack unit high, drawn at one common scale + +#### Scenario: Smooth outline +- **WHEN** the Keychain Image illustration is viewed at 200 % zoom or on a high-density display +- **THEN** its outer outline and the edges between shaded areas are smooth curves, with no visible pixel stairs + +#### Scenario: Cards stay level +- **WHEN** the root start page is shown +- **THEN** the drawn height of the Keychain illustration is not greater than that of the Gridfinity box illustration, so the Keychain card does not look larger than its neighbour + +#### Scenario: Same air on every card +- **WHEN** the start pages are shown +- **THEN** every illustration leaves a margin of at least 6 % of the card's picture field on each side, the Keychain Image illustration is not taller than the Gridfinity box illustration, and none looks larger than its neighbours in the same row + +#### Scenario: No runtime geometry +- **WHEN** the start page is opened +- **THEN** no worker is started, no WASM is loaded and no WebGL context is created + +#### Scenario: Theme tokens +- **WHEN** a colour token of the theme changes +- **THEN** the illustrations change colour with it without regenerating the image files + +#### Scenario: Crisp on any display +- **WHEN** the page is shown on a high-density display or zoomed to 200 % +- **THEN** illustration edges stay sharp + +#### Scenario: Decorative +- **WHEN** a screen reader reads a card +- **THEN** the illustration is skipped and the card is announced by its title and description + +#### Scenario: Weight +- **WHEN** the production build is made +- **THEN** each illustration is at most 60 KB uncompressed + +## ADDED Requirements + +### Requirement: Card texts +Every product and mode card SHALL have a localized description and exactly two short tags. The description SHALL say what the part is and name the special input when there is one (a photo, an image, a device); each tag SHALL be at most two short words, a noun or a noun phrase. Cards SHALL NOT show status words such as "soon", "new" or "beta" as tags, and SHALL NOT use internal jargon. In Russian, terms SHALL be translated except for names of things (Gridfinity, Skadis, keystone), which SHALL be written the same way on every card. The descriptions of the product cards SHALL cover the modes that exist. + +#### Scenario: No placeholders +- **WHEN** the root start page is shown +- **THEN** no card carries a tag that says it is coming soon, and the Keychain card describes what it makes + +#### Scenario: Rack card covers its modes +- **WHEN** the root start page is shown +- **THEN** the Rack card's description mentions panels, enclosures and shelves + +#### Scenario: Cutout card +- **WHEN** the Gridfinity start page is shown +- **THEN** the Cutout card says that pockets can be made from a photo of the tools on a sheet of paper as well as from an STL, a circle or a rectangle + +#### Scenario: Two short tags +- **WHEN** any card is shown +- **THEN** it has exactly two tags of at most two words each, in English and in Russian + +#### Scenario: Same term everywhere +- **WHEN** the Russian cards are read +- **THEN** a name such as keystone is written the same way on every card where it occurs + +### Requirement: Product colour in illustrations and logo +The start-page illustrations and the logo in the top bar SHALL take the hue of their product: blue for Gridfinity, green for Keychain and yellow for Rack. Only the hue of the illustrations and of the logo SHALL differ; the colours of the interface (accent, buttons, borders, focus, text, banners) and of the 3D viewport SHALL be the same in every product. The logo SHALL be blue where no product is active. The illustrations SHALL keep following colour tokens. + +#### Scenario: Cards by product +- **WHEN** the root start page is shown +- **THEN** the Gridfinity card illustration is blue, the Keychain one green and the Rack one yellow + +#### Scenario: Mode cards +- **WHEN** a product's start page is shown +- **THEN** all its mode illustrations have that product's hue + +#### Scenario: Logo +- **WHEN** the user opens a product +- **THEN** the logo in the top bar has that product's hue, and on the root page it is blue + +#### Scenario: Interface unchanged +- **WHEN** the user moves between products +- **THEN** buttons, selected states, focus rings and links keep the same colours + +#### Scenario: Token driven +- **WHEN** a hue token changes +- **THEN** the illustrations of that product change with it without regenerating the image files + diff --git a/openspec/changes/36-site-refresh/specs/site-identity/spec.md b/openspec/changes/36-site-refresh/specs/site-identity/spec.md new file mode 100644 index 0000000..ddc1991 --- /dev/null +++ b/openspec/changes/36-site-refresh/specs/site-identity/spec.md @@ -0,0 +1,22 @@ +# Spec Delta + +## ADDED Requirements + +### Requirement: README accuracy +The README SHALL describe the app as it is: every product with its modes and their main options, the beta status with where the version and the changelog are, a privacy note saying that the user's work and files stay in the browser and that visit counting happens only after the user accepts it, the commands, the layout of the folders that exist, a link to the roadmap, the references and the licence. Its motivation text SHALL be identical to the one on the site. The statement that there is no server and no account SHALL be followed by the privacy note. + +#### Scenario: Products and modes +- **WHEN** the README is read +- **THEN** its product table lists every mode of the app, including Cutout from a photo, Keychain Image and Shape, and Rack Panel, Enclosure and Shelf + +#### Scenario: Status and privacy +- **WHEN** the README is read +- **THEN** it says that the app is in beta, where the version and the changelog are, and that visit counting needs consent while photos and files never leave the browser + +#### Scenario: Same motivation text +- **WHEN** the motivation text of the README and of the site are compared +- **THEN** they are identical + +#### Scenario: Existing folders only +- **WHEN** the layout section of the README is read +- **THEN** every folder it lists exists in the repository diff --git a/openspec/changes/36-site-refresh/tasks.md b/openspec/changes/36-site-refresh/tasks.md new file mode 100644 index 0000000..c567c16 --- /dev/null +++ b/openspec/changes/36-site-refresh/tasks.md @@ -0,0 +1,26 @@ +# Tasks + +## 1. Texts + +- [ ] 1.1 Settle the wording of the card descriptions and tags with the owner from the table in `design.md` (three product cards and every mode card, en and ru) and write the result into `design.md`. Verify: the owner confirms the table. +- [ ] 1.2 Update `products.*` and `start.*` in `en.json` and `ru.json` (descriptions and two tags each, no "soon", Russian terms translated, `keystone` written one way). Tests: key parity, every card has exactly two tags of at most two words, the words "soon" and "скоро" do not occur on cards, the Rack product description mentions panels, enclosures and shelves, the Cutout card mentions a photo. + +## 2. README + +- [ ] 2.1 Rewrite `README.md` after the structure in `design.md`: status line with the version and the changelog link, the motivation text identical to `site.motivation`, the full product and mode table, the privacy note, the commands, the layout of existing folders, the roadmap link, the references. Tests: the motivation text in the README equals the site's (extend `site.test.ts`), every folder named in the layout exists, the README names every mode of `app/modes.tsx`. + +## 3. Illustrations + +- [ ] 3.1 Keychain card: add `scripts/art/ring.jpg` (the PoC `ring.jpeg`), make `npm run art` render it through `generate('keychain')` (decoded with `jpeg-js`, size about 50 mm, the large area open and the small ones filled), drop the paw from the script (keep `paw.ts` until unused), and scale it with `KEYCHAIN_ART_RATIO` (about 0.8 of the box drawing's height, set by eye against the Gridfinity cards). Verify: `npm run art` writes `keychainImage.svg`, the weight is within 60 KB, the card looks level with the Gridfinity cards in the browser (en and ru, 390 px). +- [ ] 3.2 Rack cards at 1 U: panel `units: 1` with the Skadis pattern showing, a small device (about 25 mm high) in the enclosure with one keystone, shelf `units: 1`; the three drawn in one group at one scale with the Gridfinity margin. Verify: `npm run art` writes the three files within 60 KB, all three show a 1 U part, compared in the browser. +- [ ] 3.2a Colour by product: `Product.hue`, the two extra tone ladders and the logo colours in `theme.css`, `ModeArt` setting `data-hue` from the art id, `Logo` with a `hue` prop, `Header` passing the product's hue (blue on the root page). Tests: each art id maps to the right hue, the wrapper carries `data-hue`, the SVG files are unchanged (no colour literals, only classes), the logo gets the hue of the active route and blue without a product, the three ladders define all seven tones, the accent, focus and button tokens do not depend on the hue. Verify: compare the root page and the three start pages in the browser (Gridfinity blue, Keychain green, Rack yellow), the header logo on each product; tune the ladder values by eye. +- [ ] 3.3 `npm run art` writes `src/app/art/manifest.json` (every illustration's drawn width and height as fractions of the 250 × 165 field); a test asserts at least 6 % free space on each side of every illustration and that the Keychain Image drawing is not taller than the Gridfinity box. Update the art paragraph of `CLAUDE.md`. Tests: the margin test, the existing weight test. + +## 4. Version + +- [ ] 4.1 Bump to 1.7.1: `package.json` and a `CHANGELOG.md` entry `1.7.1 — 36-site-refresh`. Test: the existing version test passes with the new number. + +## 5. Verification + +- [ ] 5.1 `npm test` and `npm run build` are green, the dev server starts, no console errors. +- [ ] 5.2 In the browser, en and ru and at 390 px: the root page and the three product start pages show the new texts and tags, no placeholder, the Keychain Image card shows the ring emblem with its hole open and the Rack cards show 1 U parts, the card rows look level and have the same air, the pictures and the logo are blue, green and yellow for Gridfinity, Keychain and Rack while buttons, accents and focus rings keep their colours, the README reads correctly on GitHub. diff --git a/openspec/changes/37-resource-monitor/.openspec.yaml b/openspec/changes/37-resource-monitor/.openspec.yaml new file mode 100644 index 0000000..f56fe13 --- /dev/null +++ b/openspec/changes/37-resource-monitor/.openspec.yaml @@ -0,0 +1,2 @@ +schema: spec-driven +created: 2026-10-23 diff --git a/openspec/changes/37-resource-monitor/design.md b/openspec/changes/37-resource-monitor/design.md new file mode 100644 index 0000000..2bea3be --- /dev/null +++ b/openspec/changes/37-resource-monitor/design.md @@ -0,0 +1,45 @@ +# Design + +## Context + +`31a` adds the shared activity log with jobs (`generate`, and later `ai` and `model` jobs from `32`), the status bar line and the log panel. `32` adds the ML runtime (a worker running ONNX Runtime Web on WASM or WebGPU), the registry with the backend of each model, the model store (Cache API) and the model manager. The viewport draws with three.js on WebGL, on demand. The geometry runs in a Manifold WASM worker. + +## What a page can and cannot know + +Checked in Chrome 152 on Apple Silicon (the owner's device): `PressureObserver` with source `cpu`; `longtask` and `long-animation-frame` entry types; the WebGPU adapter with `timestamp-query`, `shader-f16` and `subgroups` and `adapter.info` (vendor `apple`, architecture `metal-3`); `EXT_disjoint_timer_query_webgl2`; `performance.memory`; `navigator.hardwareConcurrency` (10) and `deviceMemory` (16). Not available: `measureUserAgentSpecificMemory` (needs cross-origin isolation, which GitHub Pages does not give), any GPU or per-core CPU utilisation, VRAM, temperature. Firefox and Safari lack Compute Pressure, timer queries and `performance.memory`, so the view degrades to what the app itself measures. + +## Goals / Non-Goals + +**Goals:** +- For the two heavy cases (rebuild and draw of a model, AI run on WASM or WebGPU) show honest numbers with their provenance. +- Help the user choose a backend and a model, and understand a slow run. + +**Non-Goals:** +- Real utilisation percentages, frame rate, automatic adaptation, telemetry. + +## Decisions + +- **Provenance on every value.** `ResourceValue = {value, unit, source: 'measured' | 'estimated' | 'unavailable', note?}`; the UI renders a small label and a tooltip ("estimated: threads × busy time"), and the copy text keeps them. A collector that the browser does not support returns `unavailable` with the reason; the view never hides a row silently. +- **Job accounting (all browsers).** `modelClient` and `mlClient` already know when a request starts and when its reply arrives; the jobs of `31a` carry `{start, end, backend, threads}` and the report is derived: duration, busy share of the worker over the job (busy time of the request over the wall time of the job, since a worker handles one request at a time) and, for WASM with `n` threads, an estimated core-equivalent load `busy × n` capped at `n` (labelled estimated: ORT's internal threads are not observable). For the editor: triangles of the result and the rebuild time. +- **Main thread.** A `PerformanceObserver` for `longtask` (and `long-animation-frame` where present) accumulates blocked time; the view shows "main thread blocked X % of the last 10 s" and the longest block, which tells whether the page stayed responsive while a job ran. Chrome and Edge only; elsewhere unavailable. +- **System CPU pressure.** `new PressureObserver(cb, {sampleInterval: 1000}).observe('cpu')` in secure contexts (HTTPS, localhost) shows the level and the time spent in each level during a job; it is a coarse, whole-system signal (the label says so) and is created only while the Resources view is open or a job runs, to avoid keeping an observer around. +- **GPU for AI on WebGPU.** The adapter's `info` and `limits` are read once (name, architecture, `maxBufferSize`). In diagnostics mode the runtime's WebGPU profiling (timestamp queries; ORT Web's `env.webgpu.profiling` with an `ondata` callback, to be verified against the pinned ORT version in task 1) accumulates kernel start and end times; the report shows the GPU time of the run and the five slowest kernels, and the ratio of GPU time to wall time. The profiling slows the run, so it is off by default; when the feature is missing the report says unavailable. GPU memory is an estimate: bytes of the weights plus the largest activations seen in the inputs and outputs of the model (labelled estimated), never a reading. +- **GPU for the editor render.** `renderer.info` (triangles, draw calls, geometries, textures) is exact. A timer query around the render (`EXT_disjoint_timer_query_webgl2`) gives the GPU time of the last draw where available (Chrome), read a few frames later and discarded when the disjoint flag is set; the CPU time to submit the draw is measured with `performance.now()`. Because the viewport renders on demand, these are "last render" values, not a running rate. +- **Memory.** `navigator.storage.estimate()` for the quota and the usage by the origin, with the model cache size taken from the model store; `performance.memory.usedJSHeapSize` where present; the byte length of the WASM memory of the modules we own when the runtime exposes it; the model manager of `32` shows the same cache numbers. +- **Diagnostics mode.** One persisted switch (`gridfinity.diagnostics`, key kept in the project's prefix), default off, that enables the costly collectors (kernel profiling, timer queries) and the continuous sampling of pressure and long tasks; its label says "slows down AI runs a little". +- **UI.** A "Resources" tab in the log panel of `31a`: groups CPU, GPU, Memory; a table "Last jobs" (what, backend, duration, busy share, GPU time when measured, triangles for rebuilds) taken from the job reports; values refresh once a second while a job runs and when the tab is opened; every number carries its provenance label; the browser support is summarised in one line ("Compute Pressure: not available in this browser"). On a phone the groups stack. The AI status line of `32` names the backend and threads ("WebGPU", "WASM ×4"). Copy of the log panel includes the resources report when the tab is shown. +- **Privacy.** Adapter name and architecture are shown to the user and included in the copied text only when the user presses Copy; nothing is sent anywhere, and no collector runs in the start pages. +- **Independent of the AI changes.** The job accounting, the viewport and memory parts work with `31a` alone; the GPU-for-AI part appears when `32` provides the runtime hook, so tasks are ordered to keep each part testable. + +## Risks / Trade-offs + +- [Proxies mistaken for utilisation] → provenance labels, no percentages unless measured, the wording "estimated" for core-equivalents and GPU memory. +- [Profiling changes what it measures] → diagnostics mode off by default, the label says it slows runs, kernel timing is aggregated per run. +- [APIs differ between browsers and versions] → each collector is feature-detected and returns `unavailable` with a reason; tests use fakes for both cases. +- [Observers leaking] → pressure and long-task observers exist only while needed and are disconnected on close; tested. +- [ORT's profiling hook changes between versions] → the ORT version is pinned in `32`; task 1 checks the hook on that version in the PoC before the work starts. + +## Open Questions + +- Whether to also offer a "benchmark" button that runs a fixed small inference on each backend and recommends one; left out of this change. +- Whether the AI status line should show GPU time live (needs profiling on); proposed only in the diagnostics mode. diff --git a/openspec/changes/37-resource-monitor/proposal.md b/openspec/changes/37-resource-monitor/proposal.md new file mode 100644 index 0000000..f5b023b --- /dev/null +++ b/openspec/changes/37-resource-monitor/proposal.md @@ -0,0 +1,32 @@ +# Proposal + +## Why + +Two kinds of work in PrintKit are heavy: rebuilding and drawing a model in the editor (CSG in a WebAssembly worker, then a three.js render), and running a neural model in the browser (ONNX Runtime Web on WASM threads or on WebGPU). The user cannot tell what the machine is doing: whether a slow AI run is slow because of the WASM backend, because the GPU is busy, because the browser throttles, or because the model is simply big; whether a heavy model makes the editor sluggish; how much memory the loaded models take. The choice between WASM and WebGPU models (`32`) is made blind. + +A web page cannot read the real load of the CPU or the GPU (there is no such API, on purpose). It can measure honest proxies for exactly these two cases, and label each as measured, estimated or unavailable. This change shows them. + +## What Changes + +- **A "Resources" view** in the activity log panel of `31a-status-log` (a tab next to the log) with three groups, CPU, GPU and Memory, plus the report of the last jobs. Each value says whether it is measured, estimated or not available in this browser. Nothing is shown as a percentage unless it is one. +- **Per-job report.** Every job of the log (model rebuild, AI run, model download) gets a short resource line when it ends: duration, backend (WASM threads or WebGPU), thread count, how much of the time the worker was busy, and for the editor the triangles drawn. The AI status line names the backend ("WebGPU", "WASM ×4"). +- **CPU.** Worker busy share over the job (an estimate of load: threads × busy time), main-thread blocked time (long tasks) to say whether the page stayed responsive, and the system CPU pressure level (nominal, fair, serious, critical) where the browser offers it (Chrome and Edge). No per-core percentages. +- **GPU.** For AI on WebGPU, GPU time of the run from the runtime's profiling (timestamp queries) in a **diagnostics mode**, with the slowest kernels; the adapter's name and architecture; an estimate of the GPU memory a model needs (weights plus tensors). For the editor's render, the cost of a draw (triangles, draw calls, CPU submit time and, where the browser supports it, GPU time from a timer query). No GPU load percentage. +- **Memory.** Size of the model files in the cache and the browser's quota (`storage.estimate`), the JS heap where the browser reports it, the memory of the WASM modules we own when the runtime exposes it. +- **Diagnostics mode.** A switch, off by default, that turns on the profiling that costs speed (kernel timing, timer queries) and says so next to the switch. +- **No FPS.** The editor does not animate, so frame rate is not shown. +- **Local only.** All values stay on the device; the report is copied only when the user presses Copy. + +## Capabilities + +### New Capabilities +- `resource-monitor`: the resources view, per-job resource reports and the diagnostics mode. + +### Modified Capabilities + +## Impact + +- New `src/app/resources/` (React-free collectors: `jobs.ts` busy accounting, `pressure.ts` Compute Pressure, `longTasks.ts`, `gpuInfo.ts`, `memory.ts`, `render.ts` for the viewport, a small `ResourceSnapshot` type with `measured | estimated | unavailable` per value), a `ResourcesTab` in the log panel of `31a`, the job interface of `31a` gets an optional `report` field, `modelClient` and `mlClient` report busy time and backend, the ML runtime of `32` gets an optional profiling hook, `Viewport` reports `renderer.info` and an optional timer query, strings in en and ru, tests. +- Out of scope: frame rate and anything about animation, real CPU or GPU load percentages (not available to a page), temperature, power and battery, per-process OS numbers, sending any measurement anywhere, adapting the app automatically to the load (choosing a model or a backend by itself), a benchmark suite. +- Version 1.8.0 (minor: a new capability). +- Depends on: `31a-status-log` (the log, jobs, status bar and panel), `32-photo-trace-ai` (the ML runtime, backends and model store), `29-beta-label` (the versioning rule). diff --git a/openspec/changes/37-resource-monitor/specs/resource-monitor/spec.md b/openspec/changes/37-resource-monitor/specs/resource-monitor/spec.md new file mode 100644 index 0000000..99a4260 --- /dev/null +++ b/openspec/changes/37-resource-monitor/specs/resource-monitor/spec.md @@ -0,0 +1,125 @@ +# Spec Delta + +## Purpose +Defines the resources view that shows, with honest labels, how the heavy work of the app, rebuilding and drawing a model and running AI models on WASM or WebGPU, uses the processor, the graphics card and the memory. + +## ADDED Requirements + +### Requirement: Resources view +The log panel SHALL have a Resources view with groups for the processor, the graphics card and the memory, and a list of the last jobs with what they were, the backend, the duration and their resource figures. The view SHALL refresh while a job runs and when it is opened. It SHALL NOT show the frame rate. + +#### Scenario: Open +- **WHEN** the user opens the log panel and chooses Resources +- **THEN** the processor, graphics and memory groups and the list of recent jobs are shown + +#### Scenario: Live while a job runs +- **WHEN** an AI run is in progress and the view is open +- **THEN** its values are refreshed about once a second + +#### Scenario: Recent jobs +- **WHEN** a model has been rebuilt and an AI run has finished +- **THEN** each appears in the list with its duration and backend, and the rebuild with its triangle count + +### Requirement: Honest figures +Every value in the view SHALL say whether it was measured, estimated or is not available in this browser, with a short reason for the last two. A value SHALL NOT be shown as a percentage of the processor or of the graphics card unless it is one that was measured. A source that the browser does not offer SHALL be listed as not available, not left out. + +#### Scenario: Measured and estimated +- **WHEN** the busy share of a WASM run with four threads is shown +- **THEN** it is labelled as an estimate and its tooltip says how it was derived + +#### Scenario: Not available +- **WHEN** the browser has no Compute Pressure +- **THEN** the system pressure row says that it is not available in this browser + +#### Scenario: No invented load +- **WHEN** the view is shown on any browser +- **THEN** it does not show a CPU or GPU load percentage that was not measured + +### Requirement: Job resource report +Every model rebuild, AI run and model download SHALL end with a short resource report in the activity log: its duration, the backend (WASM with its number of threads, or WebGPU), how much of the time the worker was busy, and for a rebuild the number of triangles. The AI status line SHALL name the backend and the number of threads. + +#### Scenario: Rebuild report +- **WHEN** a model rebuild ends +- **THEN** the log line shows how long it took and how many triangles the result has + +#### Scenario: AI report +- **WHEN** an AI run on WebGPU ends +- **THEN** the log line shows the duration and that the backend was WebGPU + +#### Scenario: Backend in the status line +- **WHEN** an AI run on WASM with four threads is in progress +- **THEN** its status line names WASM and four threads + +### Requirement: Processor figures +The view SHALL show, for the processor, the busy share of the worker over a job (an estimate of the load, counted with the number of threads), the time the main thread was blocked and the longest block, so the user can tell whether the page stayed responsive, and, where the browser offers it, the system's processor pressure level and the time spent at each level during a job, labelled as a whole-system signal. + +#### Scenario: Responsiveness +- **WHEN** a job has run for a while +- **THEN** the view shows how much of the last seconds the main thread was blocked and the longest block + +#### Scenario: System pressure +- **WHEN** the browser offers processor pressure and a job runs +- **THEN** the current level is shown and labelled as concerning the whole computer + +### Requirement: Graphics figures +For an AI run on WebGPU the view SHALL show the adapter's name and architecture and an estimate of the graphics memory the model needs; in diagnostics mode, where the browser supports timestamp queries, it SHALL also show the GPU time of the run, its share of the run's duration and the five slowest kernels. For the editor's drawing it SHALL show the number of triangles and draw calls of the last render, the time to submit it and, where the browser supports a timer query, the GPU time of the last render. + +#### Scenario: Adapter +- **WHEN** WebGPU is used +- **THEN** the view shows the adapter's name and architecture, and the memory figure is labelled as an estimate + +#### Scenario: GPU time of a run +- **WHEN** diagnostics mode is on, the browser supports timestamp queries and an AI run on WebGPU finishes +- **THEN** the report shows the GPU time and the slowest kernels + +#### Scenario: Last render +- **WHEN** the editor shows a model +- **THEN** the view shows the triangles and draw calls of its last render and the time it took to submit + +#### Scenario: Unsupported +- **WHEN** the browser does not support timestamp queries or timer queries +- **THEN** the corresponding rows are listed as not available + +### Requirement: Memory figures +The view SHALL show the space taken by the downloaded models in the browser's storage and the storage quota, the JavaScript heap where the browser reports it, and the memory of the WebAssembly modules when the runtime exposes it. + +#### Scenario: Storage +- **WHEN** two models are downloaded +- **THEN** the view shows the space they take and the quota of the browser's storage + +#### Scenario: Heap +- **WHEN** the browser reports the JavaScript heap +- **THEN** it is shown, and otherwise listed as not available + +### Requirement: Diagnostics mode +The costly measurements, kernel timing of WebGPU runs and timer queries, SHALL run only when the user turns on a diagnostics switch, which SHALL be off by default, SHALL say that it slows runs down a little, and SHALL be remembered. With it off the view SHALL still show the measurements that cost nothing. + +#### Scenario: Off by default +- **WHEN** the app is opened for the first time +- **THEN** diagnostics mode is off and no profiling runs + +#### Scenario: Warned +- **WHEN** the user looks at the diagnostics switch +- **THEN** its label says that turning it on slows AI runs a little + +#### Scenario: Remembered +- **WHEN** the user turns diagnostics on and reloads +- **THEN** it is still on + +### Requirement: Local only and tidy +All figures SHALL stay on the device and SHALL be copied only when the user presses Copy. Observers and queries SHALL run only while the view is open or a job is running and SHALL be released afterwards. Nothing SHALL run on the start pages. + +#### Scenario: No network +- **WHEN** the view is used +- **THEN** no network request carries any measurement + +#### Scenario: Released +- **WHEN** the view is closed and no job is running +- **THEN** the pressure and long-task observers are disconnected + +### Requirement: Language +All text of the view, the labels, the reasons and the report lines SHALL be available in every language of the app. + +#### Scenario: Russian +- **WHEN** the interface language is Russian +- **THEN** the groups, labels, reasons and job reports are in Russian diff --git a/openspec/changes/37-resource-monitor/tasks.md b/openspec/changes/37-resource-monitor/tasks.md new file mode 100644 index 0000000..ff7b6a4 --- /dev/null +++ b/openspec/changes/37-resource-monitor/tasks.md @@ -0,0 +1,28 @@ +# Tasks + +## 1. Check on the owner's device (spike) + +- [ ] 1.1 Extend the PoC (`poc/ai/`) with a throwaway resources panel that, for the AI models of the shortlist on WASM (1 and 4 threads) and on WebGPU, records: wall time, worker busy share, long-task blocked time, Compute Pressure levels, `adapter.info`, and with ORT Web's WebGPU profiling on the pinned version the GPU time and the slowest kernels, compared with the wall time; also the cost of the profiling (run time with and without it) and `renderer.info` plus a timer query for the keychain and cutout 3D views. Record the numbers and what is not available in `poc/ai/FINDINGS.md` and in `design.md` (Open Questions). If the profiling hook is not usable on the pinned version, say so and stop for `/opsx:update`. + +## 2. Collectors (React-free) + +- [ ] 2.1 `app/resources/types.ts` (`ResourceValue` with `measured | estimated | unavailable` and a note) and `app/resources/jobs.ts`: the report of a job from its start, end, backend and threads (duration, worker busy share, estimated core-equivalent `busy × threads` capped at the threads, triangles for a rebuild). Tests: a single-request job gives busy share 1, two requests with a gap give the right share, the core-equivalent is capped, an unknown backend is reported as such, values carry their provenance. +- [ ] 2.2 `longTasks.ts` (blocked time over a window and the longest block from `longtask` and `long-animation-frame`), `pressure.ts` (`PressureObserver` for `cpu`, time per level), `gpuInfo.ts` (adapter name, architecture, limits), `memory.ts` (`storage.estimate`, `performance.memory`, model cache size from the model store of `32`), each feature-detected and returning `unavailable` with a reason. Tests with fakes for the supported and the unsupported browser, observers are disconnected on `stop()`, no collector runs before `start()`. +- [ ] 2.3 GPU time: the profiling hook of the ML runtime of `32` (kernel start and end accumulated per run, five slowest kernels, ratio to wall time) enabled only in diagnostics mode, and the viewport's `render.ts` (`renderer.info`, submit time, optional `EXT_disjoint_timer_query_webgl2` with the disjoint check). Tests: the kernel aggregation on a recorded sample, profiling is off unless diagnostics is on, a disjoint timer result is discarded, the unsupported case is reported. + +## 3. Integration + +- [ ] 3.1 Job reports in the log of `31a`: `modelClient` and `mlClient` report start, end, backend and threads; a finished job writes its resource line; the AI status line of `32` names the backend and threads. Tests: a fake rebuild and a fake AI run produce the lines with duration, backend and (for rebuilds) triangles, the status line shows "WASM ×4" and "WebGPU", English and Russian keys. + +## 4. UI + +- [ ] 4.1 `ResourcesTab` in the log panel of `31a`: groups CPU, GPU and Memory, the table of the last jobs, a provenance label with a tooltip on every value, a one-line browser support summary, the diagnostics switch with its warning, refresh once a second while a job runs and when opened, collectors started on open and stopped on close, stacked groups on a phone, the resources report included in Copy. Strings in `en.json` and `ru.json`. No artboard exists: built from `ui/` components and `theme.css` tokens, said in the commit; compare in en and ru and at 390 px. Component tests: every value shows a label, an unsupported source is listed as not available, no percentage appears for estimated values, the switch persists in `gridfinity.diagnostics`, observers are disconnected on close, Copy contains the report, key parity. + +## 5. Version + +- [ ] 5.1 Bump to 1.8.0: `package.json` and a `CHANGELOG.md` entry `1.8.0 — 37-resource-monitor`. Test: the existing version test passes with the new number. + +## 6. Verification + +- [ ] 6.1 `npm test` and `npm run build` are green, the dev server starts, no console errors. +- [ ] 6.2 In the browser, en and ru and at 390 px: rebuild a heavy model and run an AI model on WASM and on WebGPU, check the Resources view shows the durations, the backend, the busy share labelled as an estimate, the blocked main-thread time, the pressure level where Chrome offers it, the adapter, the triangles and draw calls of the last render; turn diagnostics on and see the GPU time and the slowest kernels on WebGPU and the note that runs are slower; check Safari or Firefox (if available) lists the unsupported rows as not available; no network request carries a measurement; the observers stop when the view is closed. diff --git a/openspec/changes/38-webmcp/.openspec.yaml b/openspec/changes/38-webmcp/.openspec.yaml new file mode 100644 index 0000000..7ccf7c3 --- /dev/null +++ b/openspec/changes/38-webmcp/.openspec.yaml @@ -0,0 +1,2 @@ +schema: spec-driven +created: 2026-10-25 diff --git a/openspec/changes/38-webmcp/design.md b/openspec/changes/38-webmcp/design.md new file mode 100644 index 0000000..0979660 --- /dev/null +++ b/openspec/changes/38-webmcp/design.md @@ -0,0 +1,54 @@ +# Design + +## Context + +Each mode owns its state through `usePersistedState(key, defaults, migrate)` and builds its model with `useModel(mode, params, {enabled})` (debounce 150 ms, `busy`, `pending`, last good mesh kept). Validation is pure (`validateBaseplate`, `validateBox`, `validateRackPanel`, `rackCaseLayout` errors, …) and returns i18n keys. Export settings are a module store (`useExportSettings()`, key `gridfinity.export`) and `buildExport(parts, settings, name)` is pure. The activity log of `31a` takes entries and jobs. + +WebMCP (state of the art when this was planned, checked against the Chrome documentation): `document.modelContext.registerTool({name, description, inputSchema, execute(input, {signal}), annotations})` with annotations `readOnlyHint`, `untrustedContentHint`, `consequentialHint` (all default false); unregistering by an `AbortSignal` passed at registration; `getTools()`, `executeTool()` and a `toolchange` event for discovery and testing; cross-origin exposure is opt-in. The first preview used `navigator.modelContext`. Return shape and feature detection are not formally documented, and early-preview behaviour needs Chrome flags or an origin trial. + +## Goals / Non-Goals + +**Goals:** +- Let a browser agent configure any writable mode and read the result with real validation, without touching the DOM. +- Never do anything the user did not see or cannot undo; never download silently. +- Survive API churn: one adapter file knows the browser API. + +**Non-Goals:** +- Agents driving images, files, fonts, AI; server-side MCP; supporting every agent product. + +## Decisions + +- **One adapter, many tools.** `agent/registry.ts` is the product-level model: `AgentTool = {name, description, inputSchema, annotations, run(input, ctx): Promise}`; `ToolResult` is `{ok: true, data} | {ok: false, error: {code, message, details?}}`. `agent/webmcp.ts` is the only place that touches `document.modelContext` (falling back to `navigator.modelContext` while Chrome still has it): feature detection, `registerTool` with an `AbortController` per tool, converting a `ToolResult` to the shape the browser expects (a JSON text content; the exact shape is fixed by the spike), `toolchange`, and `dispose()`. If neither object exists, `webmcp.install()` returns `null` and the host renders nothing. Everything else is testable in Node with a fake `modelContext`. +- **Tools follow the active mode.** `AgentHost` (in `AppShell`) holds the always-present tools and the set contributed by the mounted mode. A mode contributes by calling `useAgentMode(descriptor, {state, setState, model, defaults})`; unmount removes its tools. The names do not carry the mode (`set_parameters`, not `set_baseplate_parameters`): the active mode is in `get_app_state`, and fewer tools make agents more reliable. A tool called while no mode is open answers `no_active_mode` with the list of modes and `open_mode`. +- **Descriptor per mode.** `AgentModeDescriptor = {id, writable, schema, normalise(raw) → T, validate(T) → keys, describe(T) → data}`. `schema` is a JSON Schema subset (object, number with min/max/step, integer, boolean, string enum, array with max items) written by hand for the documented fields only, with descriptions in English that include units ("mm", "cells"). `normalise` is the mode's existing `migrate` plus the same nested normalisers as `usePersistedState` (so a partial patch can never crash a mode); `validate` is the mode's existing validator. Fields not in the schema are not writable: this keeps files, images and internal ids out of reach. +- **Patch semantics.** `set_parameters({patch})`: (1) check the patch against the schema (unknown field, wrong type, out of range → error with the JSON path, nothing applied); (2) deep-merge onto the current state (arrays replaced whole, `AxisSize` objects merged by key); (3) `normalise`; (4) `validate`: validation keys are *returned* but the state **is** still applied when the keys are the mode's ordinary "invalid parameters" (the UI shows them the same way and generation is skipped), so the user sees what the agent did; hard schema violations are never applied; (5) one `setState` call, so one rebuild; (6) the result carries `applied`, `normalised` differences (e.g. a value clamped), `errors` (keys + English text from `en.json`) and `warnings`. Lengths are millimetres whatever the display unit. +- **Reading the result.** `get_model_info` waits for `!pending && !busy` with a 20 s limit and returns `{ready, parts: [{label, bbox, volumeMm3, triangles}], total, errors, warnings, plate: {printer, usableBed, plates, overflow}}`; it never returns the mesh. If the build is still running at the limit it answers `ready: false` and the caller may retry. +- **Export is a request, not an action.** `request_export({format?, name?})` validates (nothing to export, errors) and opens an `AgentExportDialog` ("An agent wants to save "), whose Download button runs the normal export path. The tool resolves when the user chooses or the dialog is dismissed (`saved | declined | timeout 120 s`), so an agent cannot download in the background and a hidden tab cannot save files. `reset_parameters` uses the same confirmation. This matches the rule of `32` (nothing is downloaded silently). +- **Log and undo.** Each call writes a log entry through the `31a` API with source `agent`, tool name and a short human summary built from the patch paths (not the whole object). `agent/history.ts` keeps the last 10 `{label, before, after}` snapshots of mode state per mode; "Undo agent change" restores `before` through the same `setState` (only if the current state still equals `after`, otherwise it offers to restore anyway). The status bar marker is the `31a` job type `agent` while a call runs. +- **Switch.** `gridfinity.agent` = `{enabled: true}`; off → `dispose()`; on → reinstall. The switch sits in the log panel header (where agent activity is visible) and the Header menu link opens it. The state is read at startup before registering. +- **Security.** Tool results contain only numbers, enums, booleans, parameter names, validation texts that we wrote, and user-entered strings of writable modes (Keychain text, export file name), the tools returning the last are annotated `untrustedContentHint`. Tool descriptions are static. Nothing in the page is executed from tool input, a string never reaches `innerHTML` or a file path unsanitised (the export name goes through the existing file-name normalisation). `exposedTo` is never set, so other origins' frames cannot call the tools. Inputs are size-limited (a patch over 8 KB is rejected). A call rate over 20 per second answers `rate_limited`. +- **Stability.** The adapter reads the flavour of the API at runtime (`document.modelContext` first), tolerates both argument shapes for `execute`, and everything is behind `webmcp.install()` returning `null`. The spike (task 1.1) pins what the current Chrome does; a version change of the browser API means editing one file. + +## Tool surface (v1) + +| Tool | Flags | Purpose | +|---|---|---| +| `get_app_state` | readOnly | product, mode, units, language, writable modes, which tools exist | +| `open_mode` | – | navigate to `{product, mode}` (hash route); unknown → error with valid ids | +| `get_parameters` | readOnly, untrusted | current parameters (mm), schema of the writable subset, defaults | +| `set_parameters` | consequential | patch, validate, apply; returns errors, warnings, normalised diff | +| `reset_parameters` | consequential | asks the user to confirm, then restores defaults | +| `get_model_info` | readOnly | build result summary and plate status | +| `get_export_settings` | readOnly | format, printer, margin, gap, multi, orientation, file name | +| `set_export_settings` | consequential | patch of the same fields (printer ids from `data/printers.ts`) | +| `request_export` | consequential | ask the user to save; resolves with `saved/declined/timeout` | + +## Risks / Trade-offs + +- The API may change or never leave preview → one adapter, no effect without it, a small change if it moves; the value now is mostly agent-driven testing and an early look. +- An agent can leave a mode in an invalid state → validation keys are returned and shown, undo is one click, the model keeps the last good mesh. +- A per-mode schema can drift from the UI → each descriptor has a test that the schema and defaults agree, and a test that every writable setting of a panel is either in the schema or listed as excluded. + +## Open Questions + +- Spike results go here: exact API name and flag in the current Chrome, the shape `execute` must return, whether images can be returned (a `capture_view` tool for the 3D view is deferred until they can), how to drive the tools in tests (the browser's `executeTool`), and whether a hidden tab keeps tools registered. diff --git a/openspec/changes/38-webmcp/proposal.md b/openspec/changes/38-webmcp/proposal.md new file mode 100644 index 0000000..cc5e761 --- /dev/null +++ b/openspec/changes/38-webmcp/proposal.md @@ -0,0 +1,40 @@ +# Proposal + +## Why + +Browsers are getting agents (Chrome with Gemini, extensions, assistants) that act on a page for the user. Today such an agent has to read the DOM and click: slow, fragile, and blind to what is valid (it does not know that a 5-unit-wide rack case exceeds 222.25 mm until the page shows an error). WebMCP is a web-platform proposal (W3C community group; early preview in Chrome) in which a page declares **tools**, each with a name, description, JSON input schema and an `execute` function, and the agent calls them directly. + +PrintKit is a good fit: it is client-only, every mode is a pure function of a small serialisable parameter object in millimetres, validation already returns keys instead of text, and the result (volume, bounding box, plate arrangement) is measurable without looking at the screen. "Baseplate 4×3 with corner magnets for a Bambu A1, give me the 3MF" becomes a few calls with real validation feedback. + +The API is still moving (the entry point is `document.modelContext` in current Chrome docs and was `navigator.modelContext` in the first preview; annotations and argument shapes change between versions). This change therefore isolates the API behind one thin adapter and does nothing at all when the browser has no WebMCP. + +## What Changes + +- **Agent bridge** (`src/agent/`): a React-free registry of *tools* (name, description, JSON schema, read-only/consequential flags, handler) and an adapter that registers them with the browser's WebMCP when it exists, re-registers when the active mode changes (`toolchange`), and removes them on unmount. No WebMCP → nothing is registered and nothing changes for the user. +- **Tools, always present:** `get_app_state` (product, mode, units, language, what is writable), `open_mode` (navigate to a product/mode), `get_export_settings`, `set_export_settings`. +- **Tools, per active mode:** + - `get_parameters` (current parameters in mm, the schema of the writable subset, defaults); + - `set_parameters` (a partial patch, deep-merged, validated against the mode's schema and normalised by the mode's own `migrate`, applied atomically; returns the validation keys with English texts and the new state, or the reasons it was rejected, without changing anything); + - `reset_parameters` (asks the user to confirm in the UI); + - `get_model_info` (waits for the build to settle: parts, bounding box, volume, triangles, validation errors and warnings, plate arrangement and overflow for the current printer); + - `request_export` (does not download by itself: opens a confirmation in the page, the file is saved only when the user presses Download). +- **Writable modes in this version:** Gridfinity baseplate and box, Rack panel, case and shelf, Keychain shape. **Read-only (state, info, export):** Gridfinity cutout and Keychain image (they depend on files the user chose, which an agent cannot supply), and anything AI (`32`–`34`). +- **Agent activity is visible and reversible.** Every call is written to the activity log of `31a` ("Agent: set_parameters baseplate.x = 4 units"), the status bar shows a short "Agent" marker while a call runs, and the last 10 agent changes can be undone one by one from the log line or the panel ("Undo agent change"). +- **A switch** "Allow browser agents" (Header menu or the log panel, persisted, default on) that unregisters everything when off. +- **Safety.** Tools expose only the parameters of the current mode and the export settings: never images, photos, meshes, model files, file names of the user's disk, or storage contents. Tool results are plain data, never instructions. Parameters that hold user text (Keychain text, file name) are marked untrusted in the annotations. +- **No new dependency, no network.** The schema check is a small local validator; nothing is sent anywhere by this change. + +## Capabilities + +### New Capabilities +- `webmcp`: tools for browser agents, the agent bridge, activity log entries, undo and the switch. + +### Modified Capabilities + +## Impact + +- New `src/agent/` (`registry.ts`, `webmcp.ts` adapter, `schema.ts` validator, `patch.ts` deep merge, `tools/{app,params,model,export}.ts`, per-mode descriptors `modes/*.ts` with schema + normalise + validate, `history.ts` undo stack), an `AgentHost` mounted in `AppShell`, a small hook `useAgentMode(descriptor, state, setState, model)` called by each writable mode, a "Allow browser agents" switch and an undo control in the log panel of `31a`, an export confirmation dialog, strings in en and ru, tests. +- Modes keep ownership of their state: the bridge calls the same setters and the same `migrate` and `validate*` functions the UI uses; geometry is untouched. +- Out of scope: tools for images, photos, STL import, fonts and AI models; running or managing AI from an agent; reading or writing the mesh store; cross-origin exposure of tools (`exposedTo`); the declarative (form attribute) WebMCP API; any server side; adapting to a particular agent product. +- Version 1.9.0 (minor: a new capability). +- Depends on: `31a-status-log` (agent entries in the log, status marker), `29-beta-label` (the versioning rule). Independent of `32`–`37`, but the tools never download models and never start AI, so they stay valid when those land. diff --git a/openspec/changes/38-webmcp/specs/webmcp/spec.md b/openspec/changes/38-webmcp/specs/webmcp/spec.md new file mode 100644 index 0000000..bb6d504 --- /dev/null +++ b/openspec/changes/38-webmcp/specs/webmcp/spec.md @@ -0,0 +1,122 @@ +# Spec Delta + +## Purpose +Defines how PrintKit exposes its parametric tools to browser agents through WebMCP, with visible, reversible and consent-based actions. + +## ADDED Requirements + +### Requirement: Registration only where the browser supports it +The app SHALL register its tools with the browser's WebMCP interface when the browser provides one, and SHALL do nothing, without errors or visible changes, when it does not. All knowledge of the browser API SHALL be in one adapter. + +#### Scenario: Supported browser +- **WHEN** the app starts in a browser with WebMCP and agents are allowed +- **THEN** the always-present tools are registered, and the tools of the open mode are added + +#### Scenario: Unsupported browser +- **WHEN** the browser has no WebMCP interface +- **THEN** no tool is registered, no error is logged, and the app behaves as before + +#### Scenario: Mode changes +- **WHEN** the user moves from one mode to another +- **THEN** the tools of the old mode are removed and those of the new mode registered, and the tool list change is announced to the browser + +### Requirement: Reading the app and the mode +The tools SHALL let an agent read the product, the mode, the units, the language, the writable modes, the current parameters in millimetres with their schema and defaults, and the export settings. They SHALL NOT return images, photos, meshes, model files, stored model data or the contents of browser storage. + +#### Scenario: App state +- **WHEN** an agent calls `get_app_state` in the Rack case mode +- **THEN** it receives the product and mode, whether the mode is writable and the list of tools available + +#### Scenario: Parameters in millimetres +- **WHEN** the user's display unit is inches and an agent calls `get_parameters` +- **THEN** all lengths are returned in millimetres, with the unit named in the schema + +#### Scenario: No tool in a read-only mode for writing +- **WHEN** the open mode is Gridfinity cutout or Keychain image +- **THEN** `set_parameters` and `reset_parameters` are not offered, and reading and export remain + +### Requirement: Validated, atomic parameter changes +`set_parameters` SHALL accept a partial patch of the writable fields of the active mode, check it against the mode's schema, normalise it with the mode's own rules and apply it as one change that causes one rebuild. A patch that breaks the schema (unknown field, wrong type, out of range, too large) SHALL be rejected with the JSON path of each problem and SHALL change nothing. The result SHALL report the values the normalisation changed and the mode's validation errors and warnings, with texts. + +#### Scenario: Valid patch +- **WHEN** an agent sets the baseplate width to 4 units and the corner fasteners to magnets +- **THEN** both are applied together, one rebuild runs, and the result lists no errors + +#### Scenario: Unknown field +- **WHEN** a patch contains a field outside the schema +- **THEN** the call fails with the path of that field and the state is unchanged + +#### Scenario: Parameters invalid for the mode +- **WHEN** a patch makes the rack case wider than the rack window +- **THEN** the change is applied as if made in the panel, the panel shows the same error, the result lists the error key and text, and the last valid model is kept + +#### Scenario: Value clamped +- **WHEN** a value is outside what the mode normalises to +- **THEN** the result names the field, the value sent and the value used + +### Requirement: Model summary +`get_model_info` SHALL wait for the current build to finish, up to a limit, and return for each part its label, bounding box, volume and triangle count, the totals, the validation errors and warnings, and the plate arrangement for the chosen printer including parts that do not fit. It SHALL NOT return mesh data. + +#### Scenario: After a change +- **WHEN** an agent changes a parameter and calls `get_model_info` +- **THEN** the answer describes the model built from the new parameters + +#### Scenario: Build still running +- **WHEN** the build does not finish within the limit +- **THEN** the answer says it is not ready and the agent can ask again + +#### Scenario: Part larger than the bed +- **WHEN** a part does not fit the printer's usable bed +- **THEN** it is listed as overflowing with the sizes + +### Requirement: Exports and resets need the user +`request_export` and `reset_parameters` SHALL show a confirmation in the page and SHALL act only when the user confirms. No tool SHALL save a file, download a model, start an AI model or delete stored data by itself. A request that is not answered within two minutes SHALL end as timed out without acting. + +#### Scenario: Export confirmed +- **WHEN** an agent calls `request_export` and the user presses Download in the dialog +- **THEN** the files are saved by the normal export path and the call returns `saved` + +#### Scenario: Export declined +- **WHEN** the user dismisses the dialog +- **THEN** nothing is saved and the call returns `declined` + +#### Scenario: Nothing to export +- **WHEN** the model has errors or no parts +- **THEN** no dialog opens and the call returns the reason + +### Requirement: Agent actions are visible and reversible +Every tool call SHALL be written to the activity log with the tool and a short summary of what changed, and the status bar SHALL show that an agent call is running. The user SHALL be able to undo each of the last ten agent changes to the parameters of a mode. + +#### Scenario: Logged +- **WHEN** an agent changes a parameter +- **THEN** the log has an entry marked as from an agent that names the tool and the changed fields + +#### Scenario: Undo +- **WHEN** the user presses "Undo agent change" after an agent set the width +- **THEN** the mode's parameters return to the values before that call + +#### Scenario: State changed since +- **WHEN** the user edited the parameters after the agent's change and presses Undo +- **THEN** the app says the parameters have changed since and asks before restoring + +### Requirement: Switch for agents +The app SHALL have a persisted switch "Allow browser agents", on by default, that removes all tools when turned off and registers them again when turned on. + +#### Scenario: Turn off +- **WHEN** the user turns the switch off +- **THEN** all tools are unregistered and a later reload keeps them off + +### Requirement: Safe tool data +Tool descriptions SHALL be static. Tool inputs SHALL be size-limited and rate-limited. Tools SHALL NOT be exposed to other origins. Results that include text entered by the user SHALL be marked as untrusted content. Tool input SHALL NOT be executed, inserted as markup or used as a file path. + +#### Scenario: Oversized patch +- **WHEN** a patch is larger than the limit +- **THEN** it is rejected and nothing changes + +#### Scenario: Too many calls +- **WHEN** an agent calls tools faster than the rate limit +- **THEN** the excess calls answer that they are rate limited and change nothing + +#### Scenario: File name +- **WHEN** an agent sets the export file name to a string with path separators or markup +- **THEN** it is normalised by the existing file-name rules before use diff --git a/openspec/changes/38-webmcp/tasks.md b/openspec/changes/38-webmcp/tasks.md new file mode 100644 index 0000000..b776d91 --- /dev/null +++ b/openspec/changes/38-webmcp/tasks.md @@ -0,0 +1,31 @@ +# Tasks + +## 1. Check the browser API (spike) + +- [ ] 1.1 In a throwaway page (or a branch of `poc/`) in the owner's Chrome, register one read-only and one writing tool and drive them with the browser's own `getTools()` / `executeTool()` and with a real agent if one is available. Record in `design.md` (Open Questions): the entry point (`document.modelContext` / `navigator.modelContext`), the flag or origin trial, the shape `execute` must return and how errors are returned, `AbortSignal` unregistering, `toolchange`, whether a background tab keeps the tools, whether images can be returned, and the Chrome version. If the API cannot be used on a normal build, stop for `/opsx:update`. + +## 2. Agent core (React-free) + +- [ ] 2.1 `agent/registry.ts` (`AgentTool`, `ToolResult`, error codes), `agent/schema.ts` (JSON-Schema subset validator returning JSON paths: object, number min/max, integer, boolean, enum, array with max items, size limit) and `agent/patch.ts` (deep merge, arrays replaced, the diff of normalisation). Tests: valid, unknown field, wrong type, out of range, oversized, arrays, merge of nested objects, diff reporting. +- [ ] 2.2 `agent/webmcp.ts`: feature detection, registration with an `AbortController` per tool, result conversion as fixed by the spike, `toolchange`, `dispose()`, call rate limit. Tests with a fake `modelContext` (both entry points): no API → `install()` is `null` and nothing throws; tools registered and removed; a thrown handler becomes an error result; rate limit. +- [ ] 2.3 `agent/history.ts`: per-mode stack of the last 10 `{label, before, after}`, undo, the changed-since check. Tests. + +## 3. Descriptors and tools + +- [ ] 3.1 Descriptors for the writable modes (`agent/modes/{baseplate,box,rackPanel,rackCase,rackShelf,keychainShape}.ts`): schema of the documented fields with unit descriptions, `normalise` (the mode's migrate), `validate`, `describe`. Tests per mode: defaults satisfy the schema, a patch round-trips through `normalise`, invalid sizes give the same keys as the validator, and a coverage test that every setting of the panel is in the schema or in an explicit exclusion list. +- [ ] 3.2 Tools: `get_app_state`, `open_mode`, `get_parameters`, `set_parameters`, `reset_parameters`, `get_model_info`, `get_export_settings`, `set_export_settings`, `request_export`. Tests with fake mode handles: unknown mode, read-only modes, patch rejected leaves state unchanged, invalid-but-applied returns keys with English texts, `get_model_info` waits for `pending`, times out with `ready: false`, never returns mesh data, printer ids checked, export file name normalised, lengths in mm regardless of unit. + +## 4. App integration + +- [ ] 4.1 `useAgentMode(descriptor, handle)` and `AgentHost` in `AppShell` (tool set follows the mounted mode); each writable mode calls the hook with its state, setter, defaults and model result; the read-only modes call it with `writable: false`. Tests: mounting a mode adds its tools, switching modes swaps them, unmount removes them, a tool called during a route change answers `no_active_mode`. +- [ ] 4.2 Activity log entries and the status marker through the API of `31a` (source `agent`, a summary built from the changed paths), the "Undo agent change" control in the log line and panel, the switch "Allow browser agents" (key `gridfinity.agent`) in the log panel header with its Header-menu entry. Strings in `en.json` and `ru.json`. No artboard exists: built from `ui/` and tokens, said in the commit. Component tests: log entry text, undo restores, changed-since prompt, switch persists and unregisters, key parity. +- [ ] 4.3 `AgentExportDialog` and the reset confirmation (`ConfirmDialog`): file list and sizes, Download runs the normal export, decline and the 120 s timeout resolve the call, one dialog at a time, focus handling and Esc. Tests: saved / declined / timeout, no download without a click, nothing opens when the model has errors. + +## 5. Version + +- [ ] 5.1 Bump to 1.9.0: `package.json` and a `CHANGELOG.md` entry `1.9.0 — 38-webmcp`. Test: the existing version test passes with the new number. + +## 6. Verification + +- [ ] 6.1 `npm test` and `npm run build` are green, the dev server starts, no console errors; in a browser without WebMCP the app behaves as before. +- [ ] 6.2 In Chrome with WebMCP enabled, en and ru: list the tools, change a baseplate (size, fasteners), a box, a rack case that becomes too wide (see the error in the panel), a keychain shape; check `get_model_info` against the STL bounding box and volume of the real export; ask for an export and confirm, decline and let it time out; undo an agent change; turn the switch off and check the tools disappear; check no tool returns images or file data and no network request is made by the tools. diff --git a/package-lock.json b/package-lock.json index 2a13d61..5e91eb6 100644 --- a/package-lock.json +++ b/package-lock.json @@ -1,12 +1,12 @@ { "name": "printkit", - "version": "0.1.0", + "version": "1.0.0", "lockfileVersion": 3, "requires": true, "packages": { "": { "name": "printkit", - "version": "0.1.0", + "version": "1.0.0", "license": "MIT", "dependencies": { "@fontsource-variable/caveat": "^5.3.0", @@ -61,6 +61,7 @@ "fake-indexeddb": "^6.2.5", "jpeg-js": "^0.4.4", "jsdom": "^29.1.1", + "onnxruntime-web": "^1.30.0", "tsx": "^4.23.15", "typescript": "^7.0.2", "vite": "^8.3.4", @@ -1625,6 +1626,72 @@ "url": "https://github.com/sponsors/oxc-project" } }, + "node_modules/@protobufjs/aspromise": { + "version": "1.1.2", + "resolved": "https://registry.npmjs.org/@protobufjs/aspromise/-/aspromise-1.1.2.tgz", + "integrity": "sha512-j+gKExEuLmKwvz3OgROXtrJ2UG2x8Ch2YZUxahh+s1F2HZ+wAceUNLkvy6zKCPVRkU++ZWQrdxsUeQXmcg4uoQ==", + "dev": true, + "license": "BSD-3-Clause" + }, + "node_modules/@protobufjs/base64": { + "version": "1.1.2", + "resolved": "https://registry.npmjs.org/@protobufjs/base64/-/base64-1.1.2.tgz", + "integrity": "sha512-AZkcAA5vnN/v4PDqKyMR5lx7hZttPDgClv83E//FMNhR2TMcLUhfRUBHCmSl0oi9zMgDDqRUJkSxO3wm85+XLg==", + "dev": true, + "license": "BSD-3-Clause" + }, + "node_modules/@protobufjs/codegen": { + "version": "2.0.5", + "resolved": "https://registry.npmjs.org/@protobufjs/codegen/-/codegen-2.0.5.tgz", + "integrity": "sha512-zgXFLzW3Ap33e6d0Wlj4MGIm6Ce8O89n/apUaGNB/jx+hw+ruWEp7EwGUshdLKVRCxZW12fp9r40E1mQrf/34g==", + "dev": true, + "license": "BSD-3-Clause" + }, + "node_modules/@protobufjs/eventemitter": { + "version": "1.1.1", + "resolved": "https://registry.npmjs.org/@protobufjs/eventemitter/-/eventemitter-1.1.1.tgz", + "integrity": "sha512-vW1GmwMZNnL+gMRaovlh9yZX74kc+TTU3FObkkurpMaRtBfLP3ldjS9KQWlwZgraRE0+dheEEoAxdzcJQ8eXZg==", + "dev": true, + "license": "BSD-3-Clause" + }, + "node_modules/@protobufjs/fetch": { + "version": "1.1.1", + "resolved": "https://registry.npmjs.org/@protobufjs/fetch/-/fetch-1.1.1.tgz", + "integrity": "sha512-GpptLrs57adMSuHi3VNj0mAF8dwh36LMaYF6XyJ6JMWlVsc+t42tm1HSEDmOs3A8fC9yyeisgLhsTVQokOZ0zw==", + "dev": true, + "license": "BSD-3-Clause", + "dependencies": { + "@protobufjs/aspromise": "^1.1.1" + } + }, + "node_modules/@protobufjs/float": { + "version": "1.0.2", + "resolved": "https://registry.npmjs.org/@protobufjs/float/-/float-1.0.2.tgz", + "integrity": "sha512-Ddb+kVXlXst9d+R9PfTIxh1EdNkgoRe5tOX6t01f1lYWOvJnSPDBlG241QLzcyPdoNTsblLUdujGSE4RzrTZGQ==", + "dev": true, + "license": "BSD-3-Clause" + }, + "node_modules/@protobufjs/path": { + "version": "1.1.2", + "resolved": "https://registry.npmjs.org/@protobufjs/path/-/path-1.1.2.tgz", + "integrity": "sha512-6JOcJ5Tm08dOHAbdR3GrvP+yUUfkjG5ePsHYczMFLq3ZmMkAD98cDgcT2iA1lJ9NVwFd4tH/iSSoe44YWkltEA==", + "dev": true, + "license": "BSD-3-Clause" + }, + "node_modules/@protobufjs/pool": { + "version": "1.1.0", + "resolved": "https://registry.npmjs.org/@protobufjs/pool/-/pool-1.1.0.tgz", + "integrity": "sha512-0kELaGSIDBKvcgS4zkjz1PeddatrjYcmMWOlAuAPwAeccUrPHdUqo/J6LiymHHEiJT5NrF1UVwxY14f+fy4WQw==", + "dev": true, + "license": "BSD-3-Clause" + }, + "node_modules/@protobufjs/utf8": { + "version": "1.1.2", + "resolved": "https://registry.npmjs.org/@protobufjs/utf8/-/utf8-1.1.2.tgz", + "integrity": "sha512-b1UQwcEZ4yCnMCD8DAL1VlbvBJE9/IX4FTIp7BG1xYpf29SLazLSrqUkj4w7Y5y7cCVP6E5tcqqcI0xemPkHug==", + "dev": true, + "license": "BSD-3-Clause" + }, "node_modules/@remix-run/route-pattern": { "version": "0.22.1", "resolved": "https://registry.npmjs.org/@remix-run/route-pattern/-/route-pattern-0.22.1.tgz", @@ -2553,6 +2620,17 @@ "integrity": "sha512-go+C6vHBUuO9jH6Tj6qEZ63JEGoMSxFdmd+6QAtthGD6xumBfIz3RbNDDs2aoRVLf9naH8lk5jnWy2UHHKgYQQ==", "license": "MIT" }, + "node_modules/@types/node": { + "version": "26.6.5", + "resolved": "https://registry.npmjs.org/@types/node/-/node-26.6.5.tgz", + "integrity": "sha512-uMzsv9stDdofIZBx4UGh4Z9ZgXnGC9seRDscilBcTHAeOFk67MZg/k9bWVfO+bk4qZmG+l52hRnLwUiH2LnJTg==", + "devOptional": true, + "license": "MIT", + "peer": true, + "dependencies": { + "undici-types": "~8.9.0" + } + }, "node_modules/@types/react": { "version": "19.3.0", "resolved": "https://registry.npmjs.org/@types/react/-/react-19.3.0.tgz", @@ -3382,6 +3460,13 @@ "integrity": "sha512-tbZNuJrLwGUp3zshBtdy4W+ORxZuIh8a5ilyIEQDC5rY1f3U20JMry0Ll3WBzU58EZKsEuJFXhb5gwv8CsPvgA==", "license": "MIT" }, + "node_modules/flatbuffers": { + "version": "25.9.23", + "resolved": "https://registry.npmjs.org/flatbuffers/-/flatbuffers-25.9.23.tgz", + "integrity": "sha512-MI1qs7Lo4Syw0EOzUl0xjs2lsoeqFku44KpngfIduHBYvzm8h2+7K8YMQh1JtVVVrUvhLpNwqVi4DERegUJhPQ==", + "dev": true, + "license": "Apache-2.0" + }, "node_modules/fsevents": { "version": "2.3.3", "resolved": "https://registry.npmjs.org/fsevents/-/fsevents-2.3.3.tgz", @@ -3402,6 +3487,13 @@ "integrity": "sha512-RbJ5/jmFcNNCcDV5o9eTnBLJ/HszWV0P73bc+Ff4nS/rJj+YaS6IGyiOL0VoBYX+l1Wrl3k63h/KrH+nhJ0XvQ==", "license": "ISC" }, + "node_modules/guid-typescript": { + "version": "1.0.9", + "resolved": "https://registry.npmjs.org/guid-typescript/-/guid-typescript-1.0.9.tgz", + "integrity": "sha512-Y8T4vYhEfwJOTbouREvG+3XDsjr8E3kIr7uf+JZ0BYloFsttiHU0WfvANVsR7TxNUJa/WpCnw/Ino/p+DeBhBQ==", + "dev": true, + "license": "ISC" + }, "node_modules/html-encoding-sniffer": { "version": "6.0.0", "resolved": "https://registry.npmjs.org/html-encoding-sniffer/-/html-encoding-sniffer-6.0.0.tgz", @@ -3823,6 +3915,13 @@ "url": "https://opencollective.com/parcel" } }, + "node_modules/long": { + "version": "5.3.2", + "resolved": "https://registry.npmjs.org/long/-/long-5.3.2.tgz", + "integrity": "sha512-mNAgZ1GmyNhD7AuqnTG3/VQ26o760+ZYBPKjPvugO8+nLbYfX6TVpJPseBvopbdY+qpZ/lKUnmEc1LeZYS3QAA==", + "dev": true, + "license": "Apache-2.0" + }, "node_modules/lru-cache": { "version": "11.5.3", "resolved": "https://registry.npmjs.org/lru-cache/-/lru-cache-11.5.3.tgz", @@ -3977,6 +4076,28 @@ "node": ">=12.20.0" } }, + "node_modules/onnxruntime-common": { + "version": "1.30.0", + "resolved": "https://registry.npmjs.org/onnxruntime-common/-/onnxruntime-common-1.30.0.tgz", + "integrity": "sha512-7fdVWjAID1dVhH/G8qK3APARunV4VkBFoCQAP7qp4Wkab0mrorvmc+sqiT+mKXOzDqdjN5j+/Z9nb4gzNPWcyA==", + "dev": true, + "license": "MIT" + }, + "node_modules/onnxruntime-web": { + "version": "1.30.0", + "resolved": "https://registry.npmjs.org/onnxruntime-web/-/onnxruntime-web-1.30.0.tgz", + "integrity": "sha512-q0y+JrrtukXSzsBWEMccVfqX25LRmosXHF+CaRJmg8pZClzcV7svNc4rKY3jL02Vb7QmRMDs1SigqR4CXAfKYQ==", + "dev": true, + "license": "MIT", + "dependencies": { + "flatbuffers": "^25.1.24", + "guid-typescript": "^1.0.9", + "long": "^5.2.3", + "onnxruntime-common": "1.30.0", + "platform": "^1.3.6", + "protobufjs": "^7.2.4" + } + }, "node_modules/parse5": { "version": "8.0.1", "resolved": "https://registry.npmjs.org/parse5/-/parse5-8.0.1.tgz", @@ -4023,6 +4144,13 @@ "url": "https://github.com/sponsors/jonschlinkert" } }, + "node_modules/platform": { + "version": "1.3.6", + "resolved": "https://registry.npmjs.org/platform/-/platform-1.3.6.tgz", + "integrity": "sha512-fnWVljUchTro6RiCFvCXBbNhJc2NijN7oIQxbwsyL0buWJPG85v81ehlHI9fXrJsMNgTofEoWIQeClKpgxFLrg==", + "dev": true, + "license": "MIT" + }, "node_modules/postcss": { "version": "8.5.29", "resolved": "https://registry.npmjs.org/postcss/-/postcss-8.5.29.tgz", @@ -4072,6 +4200,30 @@ "integrity": "sha512-AvPcP7XECNWy4LGmFQ77k7un4lSKM4eS29PTvW4ck95uYeLxXPWJM7hLuBqK91FaHqCcgJvIUCuNJjjxKE7VKQ==", "license": "MIT" }, + "node_modules/protobufjs": { + "version": "7.6.6", + "resolved": "https://registry.npmjs.org/protobufjs/-/protobufjs-7.6.6.tgz", + "integrity": "sha512-dYDWdjSl5RNb7SgPxGQcRU+GtvP7s2fpkrY0r432PcOIaZ0/rBcxEZnQN67iJhFuQiVw754JDoPruPCNdGsbjg==", + "dev": true, + "hasInstallScript": true, + "license": "BSD-3-Clause", + "dependencies": { + "@protobufjs/aspromise": "^1.1.2", + "@protobufjs/base64": "^1.1.2", + "@protobufjs/codegen": "^2.0.5", + "@protobufjs/eventemitter": "^1.1.1", + "@protobufjs/fetch": "^1.1.1", + "@protobufjs/float": "^1.0.2", + "@protobufjs/path": "^1.1.2", + "@protobufjs/pool": "^1.1.0", + "@protobufjs/utf8": "^1.1.1", + "@types/node": ">=13.7.0", + "long": "^5.3.2" + }, + "engines": { + "node": ">=12.0.0" + } + }, "node_modules/punycode": { "version": "2.3.1", "resolved": "https://registry.npmjs.org/punycode/-/punycode-2.3.1.tgz", @@ -4535,6 +4687,13 @@ "node": ">=20.18.1" } }, + "node_modules/undici-types": { + "version": "8.9.0", + "resolved": "https://registry.npmjs.org/undici-types/-/undici-types-8.9.0.tgz", + "integrity": "sha512-KTDyRTYX8sWmKXAikPHHSyc63CRPETMctyjKFupcC6OBLXT3xsN0e9aF7m+mIXutFWpUXuedtowG7iLOzp0kQg==", + "devOptional": true, + "license": "MIT" + }, "node_modules/uniq": { "version": "1.0.1", "resolved": "https://registry.npmjs.org/uniq/-/uniq-1.0.1.tgz", diff --git a/package.json b/package.json index 41ee447..0b82912 100644 --- a/package.json +++ b/package.json @@ -11,7 +11,8 @@ "dev": "vite", "build": "tsc --noEmit && vite build", "test": "vitest run", - "art": "tsx scripts/render-art.ts" + "art": "tsx scripts/render-art.ts", + "poc:ai": "vite --config poc/ai/vite.config.ts" }, "dependencies": { "@fontsource-variable/caveat": "^5.3.0", @@ -66,6 +67,7 @@ "fake-indexeddb": "^6.2.5", "jpeg-js": "^0.4.4", "jsdom": "^29.1.1", + "onnxruntime-web": "^1.30.0", "tsx": "^4.23.15", "typescript": "^7.0.2", "vite": "^8.3.4", diff --git a/poc/ai/FINDINGS.md b/poc/ai/FINDINGS.md new file mode 100644 index 0000000..7a6947a --- /dev/null +++ b/poc/ai/FINDINGS.md @@ -0,0 +1,119 @@ +# PoC нейросетевых силуэтов: результаты (2026-10-17) + +Запуск: `npm run poc:ai` → http://localhost:5198 (`POC_COI=1` включает COOP/COEP). Снимки владельца 960×1280 (`samples/`), лист находится `detectPaper`, выпрямление 4 px/мм (840×1188), ONNX Runtime Web 1.30 в воркере (`wasm.proxy`), одна нить, машина 10 ядер, WebGPU доступен. Эталон сравнения — классический `segment()` (3 силуэта на обоих фото). + +| Модель (лицензия, вес) | Бэкенд | Фото | Силуэтов | Δ размеров vs классика | IoU | Инференс | +|---|---|---|---|---|---|---| +| U²-Net-P (Apache-2.0, 4.6 МБ), вход 320 | WASM 1 нить | flash | 3 | ≤ 0.25 мм | 0.984 | ~0.85 с | +| U²-Net-P | WASM 1 нить | noflash | 3 | кусачки: ширина −27 мм (потеряна часть синей ручки), остальные ≤ 0.75 мм | 0.832 | ~0.85 с | +| BiRefNet-lite fp16/fp32 (MIT), вход 1024 (фиксированный) | WebGPU и WASM | flash | — | `std::bad_alloc` в `OrtRun` на обоих бэкендах | — | не получено | +| ISNet general int8 (Apache-2.0, 46 МБ), вход 1024 | WASM 1 нить | flash | — | инференс > 4 мин, прерван | — | не получено | +| RMBG-1.4 int8 (Bria, non-commercial) | — | — | — | не запускалась (лицензия) | — | — | + +## Выводы +1. Бэкенд: ORT Web, один воркер, WebGPU → WASM. Для лёгкой модели WASM в одну нить достаточен (< 1 с на лист), COOP/COEP не нужны. +2. BiRefNet-lite с входом 1024 в этом окружении не запустился ни на WebGPU, ни на WASM (нехватка памяти). Вход у ONNX зафиксирован 1024, уменьшить нельзя. Либо искать/конвертировать вариант с входом 512, либо проверять на другом устройстве в обычном Chrome (память Browser pane могла быть причиной). +3. ISNet int8 на 1024 px и одной нити непригоден по скорости на WASM. +4. U²-Net-P: на снимке со вспышкой почти идентичен классике, на снимке без вспышки теряет часть тёмной/синей ручки: именно тот случай, где нужен гибридный край и сравнение с классикой, и где важна модель посильнее. +5. Нюансы реализации: при `wasm.proxy` тензоры и буфер модели передаются (transfer) и становятся detached: тензор создаётся заново на каждый запуск, буфер модели нельзя использовать повторно; без proxy инференс блокирует страницу. +6. Лицензия RMBG: карточки Hugging Face — `license: other` (bria-rmbg-1.4/2.0, «source-available for non-commercial use»), RMBG-2.0 к тому же gated. + +## Прогон владельца в обычном Chrome (flash, 1 нить) + +| Модель | Бэкенд | Результат | Инференс | +|---|---|---|---| +| BiRefNet-lite fp16 / fp32 | WebGPU и WASM | `std::bad_alloc` и в обычном Chrome: не пригодна через ORT Web на 1024 | — | +| RMBG-1.4 int8 (non-commercial) | WASM | «выглядит хорошо» (лучшее качество) | 6.3 с | +| RMBG-1.4 int8 | WebGPU | то же | 14.7 с (медленнее WASM) | +| ISNet general int8 | WASM / WebGPU | кусачки потеряны, 2 из 3 предметов хорошо | 6.4 с / 6.6 с | +| U²-Net-P | WASM | быстро, качество не лучше классики | 0.82 с | +| U²-Net-P | WebGPU | ошибка: MaxPool `ceil_mode` не реализован в WebGPU-ядре | — | + +## Что отсюда следует +1. BiRefNet-lite отпадает как основной вариант (ORT Web, вход 1024, не хватает памяти на обоих бэкендах). +2. WebGPU не даёт выигрыша: для int8-моделей он медленнее или равен WASM, а часть операторов не поддержана. Бэкенд по умолчанию: WASM; WebGPU только как опция после замера. +3. Единственная модель с хорошим результатом, RMBG-1.4, имеет non-commercial лицензию. Разрешённые (MIT/Apache) модели либо не запускаются (BiRefNet), либо хуже классики (ISNet теряет предмет, U²-Net-P не лучше). +4. Потеря целого предмета (ISNet, кусачки) не лечится гибридным краем, потому что тот только уточняет границу уже найденной маски. Если ИИ-движок остаётся, объединение «классика ∪ сеть» или проверка классических силуэтов сетью надёжнее, чем сеть как единственный источник. + +## Второй раунд, прогон владельца (flash, 1 нить) + +| Модель (лицензия) | WebGPU | WASM | Качество | +|---|---|---|---| +| InSPyReNet SwinB fp16 (MIT, 199 МБ) | **1.9 с** | 78 с, **пустая маска** (fp16 на CPU не работает) | хорошо | +| ormbg int8 / fp16 (Apache-2.0, 44 / 88 МБ) | ошибка `ceil_mode` MaxPool | 19.7 с / 15.1 с | хорошо | +| ISNet imgly fp16 / fp32 (MIT, 88 / 176 МБ) | **0.23 с / 0.30 с** | — | 2 из 3 хорошо, кусачки ловит не полностью | +| ISNet pim int8, IS-Net BW fp32 (Apache-2.0) | — | 15 с | как другие ISNet: 2 из 3 | +| U²-Net полная fp32 (Apache-2.0, 176 МБ) | ошибка `ceil_mode` | 5.5 с | 2 из 3, захватывает тень у плоскогубцев | +| MVANet q4f16 / int8 (MIT) | 5.5 с | 87 с / 129 с | **размечает весь лист**: непригодна | +| BEN2 fp16 (MIT, 219 МБ) | 3.3 с | — | **размечает весь лист**: непригодна | + +## Выводы второго раунда +1. WebGPU полезен не всегда, но для моделей без `ceil_mode` MaxPool даёт ускорение в десятки раз (ISNet imgly 0.25 с, InSPyReNet 1.9 с против 15–78 с на WASM). Значит, выбор бэкенда делается **для каждой модели отдельно**: в реестре у модели список поддерживаемых бэкендов и предпочтительный; fp16-модели работают только на WebGPU (на WASM — пустая маска или медленно). +2. Кандидаты по разрешённым лицензиям: **InSPyReNet fp16 (MIT)** — хорошее качество и ~2 с на WebGPU; **ormbg (Apache-2.0)** — хорошее качество, но только WASM, 15–20 с; **ISNet imgly fp16 (MIT)** — мгновенно, но теряет часть кусачек. Из non-commercial: **RMBG-1.4** — лучшее качество, 6 с на WASM. +3. Без WebGPU разрешённых быстрых вариантов с хорошим качеством нет: остаётся ormbg (15 с) или RMBG-1.4 (6 с). +4. Остаётся проверить: снимок noflash на этих моделях, BiRefNet-lite 512, многопоточный WASM (`POC_COI=1`) для ormbg и RMBG-1.4. + +## BiRefNet-lite 512 (MIT), прогон владельца (flash, 1 нить) + +| Вариант | WebGPU | WASM | +|---|---|---| +| fp16 (98 МБ) | пустая маска | хорошо, 9.4 с | +| fp32 (192 МБ) | пустая маска | хорошо, 9.1 с | + +Вывод: на входе 512 модель запускается и даёт хороший результат, но только на WASM (~9 с); WebGPU даёт пустую маску даже в fp32, значит дело в самом WebGPU-пути для этой модели, а не в fp16. Это первый разрешённый вариант с хорошим качеством, работающий без WebGPU. fp16 не быстрее fp32 на WASM, поэтому для WASM берётся меньший по размеру файл (fp16, 98 МБ), если качество совпадёт. + +## Решение владельца +С регулировкой порога и сглаживания маски достаточно моделей с разрешёнными лицензиями (даже ISNet даёт хороший результат), поэтому RMBG-1.4 в продукт не берётся и остаётся в PoC только как эталон качества (`referenceOnly`). Классы лицензий и согласие на non-commercial из плана убраны. + +## Лицо кота: глубина против локального контраста (2026-10) +- **Depth Anything V2 Small (518 px)** на фото кота показывает глубину, но морду, глаза и т. п. не считывает (прогон владельца): лицо занимает малую часть кадра, глубина мягкая. +- **Локальный контраст яркости** (вычесть размытие, разделить на локальное отклонение, tanh), картинки в `poc/ai/findings/`: + - весь кадр (`cat_whole_local_contrast.jpg`): тело и шерсть читаются, лицо мелкое; + - **кроп морды** (`cat_face_local_contrast.jpg`): глаза, нос, усы, полосы на лбу и ухо читаются отлично; + - **4 уровня** (`cat_face_4_levels.jpg`): тот же кроп, постеризованный на 4 тона: подходит под террасный рельеф и гравировку, лицо узнаётся. +- Вывод: для портретного брелка нужны (1) кроп области лица (масштаб), (2) тональный рельеф из локального контраста (радиус задаётся в мм детали), (3) террасы/уровни. Глубина остаётся необязательным экспериментом; стоит проверить её на кропе (`cat_face` в образцах). + +### Решение владельца (2026-10-21) +Глубина полезна, поэтому в плане `34` форма строится как смесь трёх источников (контур, глубина, детали) с долями, которые задаёт пользователь, плюс кроп области. В PoC добавлен «Комбинатор формы»: берёт маску и глубину из запусков моделей на одном снимке (режим «весь снимок») и яркость, показывает смешанную высоту и затенённый рельеф. + +### PoC пайплайна Keychain (index.html) и PoC фото-вырезов (cutout.html) +`npm run poc:ai`: `/` — Keychain по этапам (1 изображение и кроп → 2 силуэт: модель маски, яркость или весь кадр → 3 глубина: модель глубины, считается по кропу → 4 детали: локальный контраст → 5 смесь контура, глубины и деталей → 3D-рельеф на three.js с орбитой и подсветкой); `/cutout.html` — сравнение с классикой для change 32. Модели силуэта и глубины выбираются отдельно и считаются независимо; результаты хранятся до смены снимка. + +### Мягкость рельефа (PoC пайплайна) +Ступени и обрывы давали сама глубина (отвесные скачки на границах уха и головы, полосы от 4-битных весов q4f16) и бугристый локальный контраст. Смягчение, которое нужно перенести в `34`: сглаживание каждого слоя в мм (глубина 0.6, детали 0.3), гамма глубины, итоговое сглаживание (0.3 мм, внутри маски, нормированная свёртка), **предел наклона** (две огибающие конусом + среднее; 1.5 ≈ 56°) и билинейная выборка высоты в узлах сетки. Усиление деталей по умолчанию 0.7. + +### Контур без ступеней (3D) +Лесенка по краю шла от сетки: в 3D брались только ячейки, где все четыре узла внутри маски. Исправлено: край кладётся по мягкой маске (σ = 1 px) с субпиксельной точностью, внешние узлы краевых ячеек сдвигаются вдоль ребра к линии F = 0.5, стенки идут по этой линии. Это же нужно сделать в `34` при построении контура (`shapeCrossSection`-подобное сглаживание маски перед построением, а не ступенчатая граница узлов). Рабочий растр по умолчанию 800 px, узлов 320. + +### Линейный арт и зернистость кота (PoC) +- **Рисунки (`ring`, `check`)**: контур по яркости (Оцу) ровный, но «заливать пустоты» заполняло и большое отверстие кольца. Нужно: мелкие области (между штрихами) заливать, крупные открывать. В PoC порог площади (по умолчанию 150 мм²); в приложении это области из change 18 (`findHoles`, режимы follow/open/level): для источника «яркость» значение по умолчанию «открыта» для областей крупнее порога. +- **Как применяется глубина** (режим PoC «только глубина»): сырая глубина → привязка к краям фото (направляющий фильтр) → нормировка по 2–98 % внутри силуэта (ближе = выше) → гамма → сглаживание → доля в смеси. Одна глубина на коте даёт чистую гладкую форму без зерна. +- **Зернистость** на целом коте даёт слой деталей (полосы шерсти при радиусе ~1 мм). Для целого животного доля деталей 15 % и усиление 0.45, радиус 1.2 мм; детали полезны на кропе лица. + +### Контур совсем без ступеней +Ступени давали три вещи: бинарная маска (край по пикселю), резка рельефа целыми ячейками сетки (лесенка ячеек) и сетка как предел точности края. В PoC край теперь режется marching squares по изолинии 0.5 мягкой маски (вершины контура на изолинии), на рисунке «галочка» шипы и кончики без ступеней. В приложении: контур как векторный полигон (`traceAlpha`, упрощение и сглаживание с сохранением углов) и обрезка рельефа этим полигоном как `CrossSection` (как `shapeCrossSection` в режиме Shape); сетка задаёт только высоты. Записано в `34` (design, spec, задача 2.3a). + +### Детали гравировкой и векторный контур +- Бугры давало прямое «тон → высота». Режим «гравировка»: продавливаются только самые тёмные черты (глаза, нос, рот, полосы), средние тона гладкие; порог тёмного ~0.15 для кропа лица. Режимы «рельеф» и «оба» сохранены. +- Контур «вау» на рисунках: полигон с передискретизацией, поиск острых углов по повороту (50°), гаусс вдоль кривой с закреплёнными углами, резка ячеек по изолинии знакового расстояния. Острия шипов остаются острыми, кривые плавные. +- Пресеты: лицо (кроп), животное целиком, рисунок/логотип; при выборе образца пресет применяется автоматически. + +### Пресеты по результатам прогонов +- **Рисунок/логотип** (`check`, `ring`): контур по яркости + гравировка внутренних штрихов (контур 55 / детали 45, режим «гравировка», порог 0.2, усиление 1.6, радиус 0.7 мм, диапазон 1.8 мм): внутренние линии и тени рисунка становятся рельефом, а не плоской плитой. +- **Животное целиком** (`cat`): гравировка с мягким порогом 0.18, усиление 0.9, радиус 0.9 мм, доли 25 / 45 / 30, диапазон 2.8 мм: полосы шерсти как узор, тело гладкое. Порог 0.45 давал пустой гладкий рельеф. +- **Лицо** (`cat_face`): порог 0.15, усиление 1.1, доли 30 / 40 / 30, диапазон 2.4 мм. + +### PoC вырезов (cutout.html) переделан как пайплайн +Этапы: 1 фото и лист (авто-углы, формат, ориентация) → 2 силуэты (классика-baseline или модель маски с порогом и сглаживанием в мм; сравнение контуров двух движков и таблица размеров/IoU; галочки по силуэтам) → 3 контур (векторный, с сохранением углов, как в Keychain) → 4 вырез и корпус (глубина, зазор, поле, шаг юнита, высота; корпус целыми юнитами) → 3D с карманами по точному контуру (зазор сдвигает контур карманa по знаковому расстоянию), инструменты полупрозрачным слоем. Контуры карманов режутся по изолинии без ступеней. Старая страница сравнения (`runAll`) заменена этим пайплайном: сравнение осталось как галочка и таблица. + +### Почему AI-контур был ребристым и как исправлено (cutout.html) +Классика строит маску по самому фото при 4 px/мм (0.25 мм на пиксель) + размытие σ = 1 px + изолиния + упрощение 0.15 мм. Модель отдаёт 512 × 512 на весь лист (≈ 0.6 мм на пиксель), билинейное растяжение давало волну с периодом пикселя сети. Исправления: бикубическая интерполяция (Катмулл–Ром), второй проход по каждому объекту (кроп с полем 12 мм: ≈ 0.3 мм на пиксель), векторное сглаживание с сохранением углов. Метрика «шероховатость» (среднее отклонение вершин от контура, сглаженного σ = 1.5 мм): классика 59 мкм, AI без второго прохода 65 мкм, AI со вторым проходом 58 мкм (flash, birefnet-lite-512-fp16, WASM). Второй проход стоит ≈ 6 с на объект. + +### Рябь на стенках кармана в 3D PoC вырезов (исправлено) +Вертикальные полосы на стенках появлялись даже у классики и были артефактом построения 3D в PoC, а не контура: стенки собирались из точек пересечения с узлами сетки (0.7 мм) и имели плоские нормали по граням. Теперь, как в приложении (контур → смещение на зазор → выдавливание), стенка кармана строится вдоль самого полигона, верх — прямоугольник с отверстиями (`ShapeUtils.triangulateShape`), дно кармана — контур, а нормали вершин усредняются по окну ≈ 1 мм вдоль контура (поле «окно нормалей стенки»), чтобы шум контура в десятки мкм не давал полос. Число треугольников: 263 тыс. → 36 тыс. + +### Кривые Безье вместо полилинии (cutout.html) +Подгонка кубических кривых (Шнейдер) к контуру, допуск 0.15 мм, углы по повороту 50°, выборка кривой ≈ 0.5 мм, зазор по аналитическим нормалям (дуга на выпуклых углах, митра на вогнутых). Фото flash: классика 248 кривых (747 контрольных точек) вместо 8902 вершин полилинии, ИИ 304 кривые; в модели ≈ 3.2 тыс. вершин, 13 тыс. треугольников. Работает для обоих движков одинаково: этап стоит после детекции. Оформлено как change `35-smooth-outline`. + +### Допуск 0.25 мм и шаг 1 мм; контур Keychain +Владелец: допуск 0.25 и шаг 1 дают лучший результат (меньше всего ряби на шейке отвёртки) → значения по умолчанию в PoC вырезов и Keychain и в плане `35`. Контур Keychain в PoC (`index.html`) теперь тоже через кривые Безье: рисунок `check`: 3475 вершин полилинии → 40 кривых (121 контрольная точка), 262 вершины в модели; нормали стенки аналитические, поверхность остаётся полем высот, обрезанным по подогнанному контуру. diff --git a/poc/ai/cutout.html b/poc/ai/cutout.html new file mode 100644 index 0000000..c28b057 --- /dev/null +++ b/poc/ai/cutout.html @@ -0,0 +1,83 @@ + +PoC: Вырезы — пайплайн фото → карманы + +
PoC Вырезы: фото инструментов на листе → карманы в корпусе · Keychain: пайплайн → + · isolated: , cores:
+
+
+

1Фото и лист

+ + + + + +
+

2Силуэты

+ + + +
+ + + +
+
+
+

3Контур (как в keychain)

+ + + + + + + +
+

4Вырез и корпус

+ + + + + + +
+
+
+ +
+ + diff --git a/poc/ai/cutout.ts b/poc/ai/cutout.ts new file mode 100644 index 0000000..6ed5ff6 --- /dev/null +++ b/poc/ai/cutout.ts @@ -0,0 +1,361 @@ +/** + * PoC пайплайна вырезов (change 25/25a/32): фото инструментов на листе → углы листа и выпрямление → силуэты (классика или модель маски) + * → векторный контур → карманы в корпусе → 3D. Контуры и сглаживание те же, что в PoC Keychain (outline.ts). + */ +import * as THREE from 'three' +import { OrbitControls } from 'three/addons/controls/OrbitControls.js' +import { toColorImage, toGrayImage } from '../../src/features/keychain/decodeImage' +import { detectPaper } from '../../src/geometry/photo/detect' +import { guessPaper } from '../../src/geometry/photo/aspect' +import { paperSize, type Paper, type PaperOrientation } from '../../src/geometry/photo/paper' +import { rectify, type Rectified } from '../../src/geometry/photo/homography' +import { segment, type Segmentation } from '../../src/geometry/photo/segment' +import { edt } from '../../src/geometry/raster/edt' +import { gaussianBlur, MIN_SIGMA } from '../../src/geometry/raster/blur' +import { fillHoles, labelComponents, removeSpecks } from '../../src/geometry/raster/components' +import { backendsOf, CANDIDATES } from './models' +import { infer } from './engine' +import { bezTan, findCorners, fitLoop, signedDistance, tessellate, vectorOutline, type Bez, type Pt } from './outline' + +const $ = (id: string) => document.getElementById(id) as T +const num = (id: string) => +$(id).value +const chk = (id: string) => $(id).checked +const logEl = $('log') +const log = (s: string) => { logEl.textContent += s + '\n'; logEl.scrollTop = logEl.scrollHeight; console.log(s) } +$('coi').textContent = String(self.crossOriginIsolated); $('cores').textContent = String(navigator.hardwareConcurrency) +const ones = (n: number) => new Uint8Array(n).fill(1) + +for (const c of CANDIDATES.filter((x) => (x.kind ?? 'matte') === 'matte')) $('modelsel').append(new Option(c.label, c.id)) +$('modelsel').value = 'birefnet-lite-512-fp16' +function syncEp() { + const ep = $('ep'), list = backendsOf($('modelsel').value), keep = ep.value + ep.innerHTML = ''; for (const b of list) ep.append(new Option(b, b)) + ep.value = list.includes(keep as 'wasm' | 'webgpu') ? keep : list[0]; ep.disabled = list.length === 1 +} +syncEp(); $('modelsel').addEventListener('change', syncEp) + +// ---------- состояние ---------- +interface Sil { id: number; widthMm: number; heightMm: number; areaMm2: number; bbox: [number, number, number, number] } +interface Seg { labels: Int32Array; sils: Sil[] } +let photo: { w: number; h: number; rgb: Uint8Array; gray: ReturnType; color: ReturnType } | null = null +let detection: { corners: Pt[]; paper: Paper; info: string } | null = null +let rect: Rectified | null = null +let classic: Seg | null = null +let matte: Float32Array | null = null // мягкая альфа модели на выпрямленном растре +const excluded = new Set() + +async function loadPhoto() { + const f = $('file').files?.[0] + const el = new Image(); el.src = f ? URL.createObjectURL(f) : `./samples/${$('sample').value}.jpeg`; await el.decode() + const cv = document.createElement('canvas'); cv.width = el.naturalWidth; cv.height = el.naturalHeight + const g = cv.getContext('2d')!; g.drawImage(el, 0, 0) + const d = g.getImageData(0, 0, cv.width, cv.height) + const gray = toGrayImage(d.data, d.width, d.height), color = toColorImage(d.data, d.width, d.height) + photo = { w: d.width, h: d.height, rgb: color.rgb, gray, color } + const found = detectPaper(color) + if (!found) { detection = null; log('лист не найден'); $('st-paper').textContent = 'лист не найден'; return } + const guess = guessPaper(found.corners, d.width, d.height) + detection = { corners: found.corners, paper: paperSize('a4', 'portrait'), info: '' } + detection.info = `найден лист, уверенность ${found.confidence.toFixed(2)}, формат ${guess?.preset ?? '?'} / ${guess?.orientation ?? '?'}` + ;(window as unknown as { __guess: unknown }).__guess = guess + log(`фото ${d.width}×${d.height}: ${detection.info}`) + matte = null; excluded.clear(); rebuildRect(); schedule() +} + +function currentPaper(): Paper { + const guess = (window as unknown as { __guess: ReturnType }).__guess + const pp = $('ppaper').value, po = $('porient').value + const preset = pp === 'auto' ? guess?.preset ?? 'a4' : (pp as 'a4' | 'letter'), orient = (po === 'auto' ? guess?.orientation ?? 'portrait' : po) as PaperOrientation + return paperSize(preset, orient) +} + +function rebuildRect() { + if (!photo || !detection) return + const paper = currentPaper() + rect = rectify(photo.gray, detection.corners, paper, photo.color.rgb) + $('st-paper').textContent = `${detection.info}; лист ${paper.width}×${paper.height} мм` + if (!rect) { log('углы листа не дают выпрямления'); return } + const t = performance.now() + classic = segFromClassic(rect, segment(rect, { margin: num('margin'), ...(($('sens').value) ? { sensitivity: num('sens') } : {}) })) + log(`выпрямлено ${rect.image.width}×${rect.image.height} (${rect.scale} px/мм); классика: ${classic.sils.length} силуэтов за ${(performance.now() - t).toFixed(0)} мс`) + matte = null +} +const segFromClassic = (r: Rectified, s: Segmentation): Seg => ({ labels: s.labels, sils: s.silhouettes.map((x) => ({ id: x.id, widthMm: x.widthMm, heightMm: x.heightMm, areaMm2: x.areaMm2, bbox: x.bbox })) }) + +/** Силуэты по мягкой маске модели: сглаживание, порог, поле у края, заливка пустот, мелочь долой, компоненты по порядку обхода сверху вниз. */ +function aiSegmentation(r: Rectified, alpha: Float32Array): Seg { + const w = r.image.width, h = r.image.height, s = r.scale, n = w * h + const sg = num('smooth') * s, a = sg > 0 ? gaussianBlur(alpha, w, h, Math.max(MIN_SIGMA, sg), ones(n)) : alpha, thr = num('thr') + let mask: Uint8Array = new Uint8Array(n); for (let i = 0; i < n; i++) mask[i] = a[i] >= thr ? 1 : 0 + const mp = Math.round(num('margin') * s) + for (let j = 0; j < h; j++) for (let i = 0; i < w; i++) if (i < mp || j < mp || i >= w - mp || j >= h - mp) mask[j * w + i] = 0 + mask = removeSpecks(fillHoles(mask, w, h).mask, w, h, Math.round(2 * s * s)) + const cc = labelComponents(mask, w, h), bb: [number, number, number, number][] = Array.from({ length: cc.count + 1 }, () => [w, h, -1, -1]) + for (let j = 0; j < h; j++) for (let i = 0; i < w; i++) { const k = cc.labels[j * w + i]; if (k) { const b = bb[k]; if (i < b[0]) b[0] = i; if (j < b[1]) b[1] = j; if (i > b[2]) b[2] = i; if (j > b[3]) b[3] = j } } + const keep = new Int32Array(cc.count + 1), sils: Sil[] = []; let id = 0 + for (let k = 1; k <= cc.count; k++) { + const b = bb[k], wm = (b[2] - b[0] + 1) / s, hm = (b[3] - b[1] + 1) / s, area = cc.sizes[k] / (s * s) + if (area >= num('minarea') && wm >= 3 && hm >= 3) { keep[k] = ++id; sils.push({ id, widthMm: wm, heightMm: hm, areaMm2: area, bbox: b }) } + } + const labels = new Int32Array(n); for (let i = 0; i < n; i++) labels[i] = keep[cc.labels[i]] + return { labels, sils } +} + +// ---------- превью ---------- +function show(id: string, w: number, h: number, px: (i: number) => [number, number, number]) { + const cv = $(id); cv.width = w; cv.height = h + const g = cv.getContext('2d')!, im = g.createImageData(w, h) + for (let i = 0; i < w * h; i++) { const [r, gg, b] = px(i); im.data[i * 4] = r; im.data[i * 4 + 1] = gg; im.data[i * 4 + 2] = b; im.data[i * 4 + 3] = 255 } + g.putImageData(im, 0, 0) +} +function previewPhoto() { + if (!photo) return + const cv = $('pv-photo'), s = Math.min(1, 480 / photo.w); cv.width = Math.round(photo.w * s); cv.height = Math.round(photo.h * s) + const t = document.createElement('canvas'); t.width = photo.w; t.height = photo.h + const im = t.getContext('2d')!.createImageData(photo.w, photo.h) + for (let i = 0; i < photo.w * photo.h; i++) { im.data[i * 4] = photo.rgb[i * 3]; im.data[i * 4 + 1] = photo.rgb[i * 3 + 1]; im.data[i * 4 + 2] = photo.rgb[i * 3 + 2]; im.data[i * 4 + 3] = 255 } + t.getContext('2d')!.putImageData(im, 0, 0) + const g = cv.getContext('2d')!; g.drawImage(t, 0, 0, cv.width, cv.height) + if (detection) { g.strokeStyle = '#ff3'; g.lineWidth = 2; g.beginPath(); detection.corners.forEach(([x, y], i) => (i ? g.lineTo(x * s, y * s) : g.moveTo(x * s, y * s))); g.closePath(); g.stroke() } +} + +// ---------- контуры и поле ---------- +interface Outline { id: number; loops: Pt[][] } +function outlineOf(seg: Seg, sil: Sil, r: Rectified): Outline { + const w = r.image.width, h = r.image.height, PAD = 4 + const x0 = Math.max(0, sil.bbox[0] - PAD), y0 = Math.max(0, sil.bbox[1] - PAD), x1 = Math.min(w - 1, sil.bbox[2] + PAD), y1 = Math.min(h - 1, sil.bbox[3] + PAD), bw = x1 - x0 + 1, bh = y1 - y0 + 1 + const v = new Float32Array(bw * bh); for (let j = 0; j < bh; j++) for (let i = 0; i < bw; i++) if (seg.labels[(y0 + j) * w + x0 + i] === sil.id) v[j * bw + i] = 255 + const f = gaussianBlur(v, bw, bh, 0.8, ones(bw * bh)), sigma = chk('vec') ? Math.max(0, num('osm') * r.scale) : 0 + const loops = vectorOutline(Uint8Array.from(f, (x) => Math.round(x)), bw, bh, sigma, num('cang')).map((l) => l.map(([x, y]) => [x + x0, y + y0] as Pt)) + return { id: sil.id, loops } +} + +/** Шероховатость контура, мм: среднее отклонение вершин от того же контура, сглаженного гауссом σ = 1.5 мм (чем меньше, тем плавнее линия). */ +function roughness(loops: Pt[][], scale: number): number { + let sum = 0, cnt = 0 + for (const l of loops) { + if (l.length < 12) continue + const step = 0.7, n = l.length, sg = (1.5 * scale) / step, r = Math.ceil(3 * sg), k: number[] = []; let ks = 0 + for (let i = -r; i <= r; i++) { const v = Math.exp(-(i * i) / (2 * sg * sg)); k.push(v); ks += v } + for (let i = 0; i < n; i++) { let x = 0, y = 0; for (let d = -r; d <= r; d++) { const q = l[(((i + d) % n) + n) % n]; x += k[d + r] * q[0]; y += k[d + r] * q[1] } ; sum += Math.hypot(l[i][0] - x / ks, l[i][1] - y / ks); cnt++ } + } + return cnt ? sum / cnt / scale : 0 +} + +interface Field { key: string; nx: number; ny: number; cell: number; sd: Float32Array; cx: number; cy: number; binW: number; binH: number; scale: number } +/** Смещение замкнутого контура (против часовой стрелки) наружу на d мм по нормалям вершин с митрой (масштаб ≤ 2). */ +function offsetLoop(p: [number, number][], d: number): [number, number][] { + if (d <= 0) return p + const n = p.length + return p.map((b, i) => { + const a = p[(i - 1 + n) % n], c = p[(i + 1) % n] + const e1 = [b[0] - a[0], b[1] - a[1]], e2 = [c[0] - b[0], c[1] - b[1]], l1 = Math.hypot(e1[0], e1[1]) || 1, l2 = Math.hypot(e2[0], e2[1]) || 1 + const n1 = [e1[1] / l1, -e1[0] / l1], n2 = [e2[1] / l2, -e2[0] / l2], mx = n1[0] + n2[0], my = n1[1] + n2[1], ml = Math.hypot(mx, my) || 1 + const cos = (n1[0] * mx + n1[1] * my) / ml, k = d / Math.max(0.5, cos) + return [b[0] + (mx / ml) * k, b[1] + (my / ml) * k] as [number, number] + }) +} + + +function activeSeg(): Seg | null { + if ($('eng').value === 'ai' && rect && matte) return aiSegmentation(rect, matte) + return classic +} +const otherSeg = (): Seg | null => ($('eng').value === 'ai' ? classic : rect && matte ? aiSegmentation(rect, matte) : null) + +let timer = 0 +function schedule() { clearTimeout(timer); timer = window.setTimeout(update, 120) } +function update() { + for (const id of ['thr', 'smooth']) { const e = $(id); (e.nextElementSibling as HTMLElement).textContent = e.value } + $('aibox').hidden = $('eng').value !== 'ai'; $('classicbox').hidden = !$('aibox').hidden + previewPhoto() + const seg = activeSeg(); if (!rect || !seg) return + $('st-sil').textContent = `${seg.sils.length} силуэтов` + ($('eng').value === 'ai' && !matte ? ' (маска не посчитана: запустите модель)' : '') + // список силуэтов с галочками + const list = $('sils'); list.innerHTML = '' + for (const s of seg.sils) { + const l = document.createElement('label'), c = document.createElement('input'); c.type = 'checkbox'; c.checked = !excluded.has(s.id) + c.addEventListener('input', () => { c.checked ? excluded.delete(s.id) : excluded.add(s.id); schedule() }) + l.append(c, ` №${s.id}: ${s.widthMm.toFixed(1)} × ${s.heightMm.toFixed(1)} мм, ${s.areaMm2.toFixed(0)} мм²`); list.append(l, document.createElement('br')) + } + // превью силуэтов (+ контуры другого движка) + const W = rect.image.width, H = rect.image.height, other = chk('cmp') ? otherSeg() : null + show('pv-sil', W, H, (i) => { + const l = seg.labels[i], ex = excluded.has(l), o = other ? other.labels[i] > 0 : false + const gv = rect!.image.gray[i] + if (!l) return o ? [gv * 0.7 + 60, gv * 0.5, gv * 0.5] : [gv, gv, gv] + return ex ? [gv * 0.5, gv * 0.5, gv * 0.5] : [gv * 0.45, gv * 0.45 + 120, gv * 0.45 + 40] + }) + compareTable(seg, other) + build() +} +function compareTable(seg: Seg, other: Seg | null) { + const el = $('cmptab'); if (!other || !rect) { el.innerHTML = ''; return } + const n = seg.labels.length, rows = ['активный ммдругой ммΔ ширΔ высIoU'] + for (const a of seg.sils) { + const ov = new Map(); for (let i = 0; i < n; i++) if (seg.labels[i] === a.id) { const k = other.labels[i]; if (k) ov.set(k, (ov.get(k) ?? 0) + 1) } + let best = 0, bo = 0; for (const [k, v] of ov) if (v > bo) { bo = v; best = k } + const b = other.sils.find((x) => x.id === best); if (!b) { rows.push(`${a.widthMm.toFixed(1)}×${a.heightMm.toFixed(1)}не найден`); continue } + let i2 = 0, u2 = 0; for (let i = 0; i < n; i++) { const x = seg.labels[i] === a.id, y = other.labels[i] === best; if (x && y) i2++; if (x || y) u2++ } + rows.push(`${a.widthMm.toFixed(1)}×${a.heightMm.toFixed(1)}${b.widthMm.toFixed(1)}×${b.heightMm.toFixed(1)}${(b.widthMm - a.widthMm).toFixed(2)}${(b.heightMm - a.heightMm).toFixed(2)}${(i2 / u2).toFixed(3)}`) + } + el.innerHTML = '' + rows.join('') + '
' +} + +// ---------- 3D ---------- +const view = $('view') +const renderer = new THREE.WebGLRenderer({ antialias: true }); view.append(renderer.domElement) +const scene = new THREE.Scene(); scene.background = new THREE.Color(0xe8e8e4) +const camera = new THREE.PerspectiveCamera(35, 1, 1, 3000) +const controls = new OrbitControls(camera, renderer.domElement) +scene.add(new THREE.HemisphereLight(0xffffff, 0x666666, 0.55)) +const sun = new THREE.DirectionalLight(0xffffff, 2.2); scene.add(sun) +const fillLight = new THREE.DirectionalLight(0xffffff, 0.5); fillLight.position.set(60, -80, 40); scene.add(fillLight) +const mat = new THREE.MeshStandardMaterial({ color: 0xd8d2c4, roughness: 0.75, metalness: 0, side: THREE.DoubleSide }) +const toolMat = new THREE.MeshStandardMaterial({ color: 0x1fb6c9, transparent: true, opacity: 0.4, roughness: 0.5 }) +const group = new THREE.Group(); scene.add(group) +function setLight() { const a = (num('lightang') * Math.PI) / 180; sun.position.set(Math.cos(a) * 120, Math.sin(a) * 120, 100) } +function resize() { const w = view.clientWidth, h = view.clientHeight; renderer.setSize(w, h, false); camera.aspect = w / h; camera.updateProjectionMatrix() } +new ResizeObserver(resize).observe(view); resize(); setLight() +renderer.setAnimationLoop(() => { controls.update(); renderer.render(scene, camera) }) +let fitted = '' +function frame(size: number, preset: 'top' | 'iso') { + const d = size * 1.7 + if (preset === 'top') { camera.up.set(0, 1, 0); camera.position.set(0, -0.001, d) } else { camera.up.set(0, 0, 1); camera.position.set(-d * 0.45, -d * 0.75, d * 0.7) } + camera.lookAt(0, 0, 0); controls.target.set(0, 0, 0); controls.update() +} +$('vtop').addEventListener('click', () => frame(Math.max(binDims.w, binDims.h), 'top')) +$('viso').addEventListener('click', () => frame(Math.max(binDims.w, binDims.h), 'iso')) +const binDims = { w: 100, h: 100 } + +function build() { + const seg = activeSeg(); if (!rect || !seg) return + const sels = seg.sils.filter((s) => !excluded.has(s.id)) + group.clear() + if (!sels.length) { $('stat3d').textContent = 'нет выбранных силуэтов'; return } + const sc = rect.scale, cell0 = Math.max(0.15, num('cell')) + const outs = sels.map((s) => outlineOf(seg, s, rect!)) + $('st-out').textContent = `${outs.reduce((a, o) => a + o.loops.length, 0)} петель, ${outs.reduce((a, o) => a + o.loops.reduce((b, l) => b + l.length, 0), 0)} вершин; шероховатость ${(roughness(outs.flatMap((o) => o.loops), sc) * 1000).toFixed(0)} мкм` + // габарит всех инструментов (мм листа) → корпус целыми юнитами + let x0 = Infinity, y0 = Infinity, x1 = -Infinity, y1 = -Infinity + for (const o of outs) for (const l of o.loops) for (const [x, y] of l) { x0 = Math.min(x0, x); x1 = Math.max(x1, x); y0 = Math.min(y0, y); y1 = Math.max(y1, y) } + const cx = (x0 + x1) / 2 / sc, cy = (y0 + y1) / 2 / sc, ex = (x1 - x0) / sc, ey = (y1 - y0) / sc, pitch = num('pitch'), m = num('bmargin') + num('clear') + const ux = Math.max(1, Math.ceil((ex + 2 * m) / pitch - 1e-9)), uy = Math.max(1, Math.ceil((ey + 2 * m) / pitch - 1e-9)), binW = ux * pitch, binH = uy * pitch + binDims.w = binW; binDims.h = binH + $('st-bin').textContent = `инструменты ${ex.toFixed(0)}×${ey.toFixed(0)} мм → корпус ${ux}×${uy} юнитов (${binW}×${binH} мм)` + // Карманы строятся прямо по контуру (как в приложении: контур → смещение на зазор → выдавливание), а не по узлам сетки: + // стенки — вертикальные ленты вдоль полигона с гладкими нормалями, верх — прямоугольник с отверстиями-карманами, дно кармана — сам контур. + const zTop = num('blockh'), zFloor = Math.max(1, zTop - num('depth')), clearMm = num('clear') + const toMm = (l: Pt[]): [number, number][] => l.map(([x, y]) => [x / sc - cx, -(y / sc - cy)]) + const area = (l: [number, number][]) => l.reduce((a, [x, y], i) => { const [x2, y2] = l[(i + 1) % l.length]; return a + (x * y2 - x2 * y) / 2 }, 0) + interface Pocket { pts: [number, number][]; nrm: [number, number][] } // вершины (мм) и единичные нормали наружу кармана + let fitInfo = { segs: 0, ctrl: 0 } + const pockets: Pocket[] = outs.map((o) => { + const outer = o.loops.reduce((best, l) => (Math.abs(area(toMm(l))) > Math.abs(area(toMm(best))) ? l : best), o.loops[0]) + let poly = toMm(outer); if (area(poly) < 0) poly = poly.reverse() // против часовой стрелки: наружу — вправо от хода + if (!chk('fit')) { // старый путь: полилиния со шумом, нормали усредняются по окну + const pts = offsetLoop(poly, clearMm), n = pts.length, perim = pts.reduce((a, q, i) => a + Math.hypot(pts[(i + 1) % n][0] - q[0], pts[(i + 1) % n][1] - q[1]), 0), k = Math.max(1, Math.round((num('nwin') / perim) * n)) + const en = pts.map((q, i) => { const r = pts[(i + 1) % n], dx = r[0] - q[0], dy = r[1] - q[1], l = Math.hypot(dx, dy) || 1; return [dy / l, -dx / l] as [number, number] }) + const nrm = pts.map((_, i) => { let x = 0, y = 0; for (let d = -k; d <= k; d++) { const e = en[(((i + d) % n) + n) % n], w = 1 - Math.abs(d) / (k + 1); x += w * e[0]; y += w * e[1] } const l = Math.hypot(x, y) || 1; return [x / l, y / l] as [number, number] }) + fitInfo.segs += n; fitInfo.ctrl += n + return { pts, nrm } + } + // гладкие кривые Безье: минимум точек, нормали аналитические; углы остаются углами, на выпуклых углах зазор даёт дугу, на вогнутых — митру + const { segs } = fitLoop(poly, findCorners(poly, 3, num('cang')), num('ftol')) + fitInfo.segs += segs.length; fitInfo.ctrl += 3 * segs.length + 1 + const samples = tessellate(segs, num('fstep')), pts: [number, number][] = [], nrm: [number, number][] = [] + const push = (p: [number, number], nv: [number, number]) => { pts.push([p[0] + nv[0] * clearMm, p[1] + nv[1] * clearMm]); nrm.push(nv) } + let si = -1 + for (let i = 0; i < samples.length; i++) { + const sm = samples[i] + if (sm.seg !== si) { // начало нового сегмента: стык с предыдущим + if (si >= 0 && clearMm > 0) { + const c = segs[si] as Bez, nEnd = (() => { const t = bezTan(c, 1); return [t[1], -t[0]] as [number, number] })(), nS = sm.n, cr = nEnd[1] * 0 // выпуклость по знаку векторного произведения касательных + const tE = bezTan(c, 1), tS: [number, number] = [-sm.n[1], sm.n[0]], cross = tE[0] * tS[1] - tE[1] * tS[0], dot = nEnd[0] * nS[0] + nEnd[1] * nS[1] + void cr + if (dot < 0.9986) { // угол > ~3° + const P = c[3] + if (cross > 0) { // выпуклый угол: дуга радиуса зазора от нормали к нормали + const a0 = Math.atan2(nEnd[1], nEnd[0]); let da = Math.atan2(nS[1], nS[0]) - a0; while (da < 0) da += 2 * Math.PI + const steps = Math.max(1, Math.ceil(da / (Math.PI / 12))) + for (let t = 1; t < steps; t++) { const a = a0 + (da * t) / steps, nv: [number, number] = [Math.cos(a), Math.sin(a)]; push(P, nv) } + } else { const bx = nEnd[0] + nS[0], by = nEnd[1] + nS[1], bl = Math.hypot(bx, by) || 1, k = clearMm / Math.max(0.4, bl / 2); pts.push([P[0] + (bx / bl) * k, P[1] + (by / bl) * k]); nrm.push([bx / bl, by / bl]) } + } + } + si = sm.seg + } + push(sm.p, sm.n) + } + return { pts, nrm } + }) + const hx = binW / 2, hy = binH / 2, pos: number[] = [], nor: number[] = [] + const tri = (a: number[], b: number[], c: number[], n: number[]) => { pos.push(...a, ...b, ...c); for (let t = 0; t < 3; t++) nor.push(...n) } + const flat = (poly: [number, number][], holes: [number, number][][], z: number, up: boolean) => { + const pts = [...poly, ...holes.flat()], faces = THREE.ShapeUtils.triangulateShape(poly.map((p) => new THREE.Vector2(p[0], p[1])), holes.map((h) => h.map((p) => new THREE.Vector2(p[0], p[1])))) + for (const [i, j, k] of faces) { + const A = pts[i], B = pts[j], C = pts[k], sa = (B[0] - A[0]) * (C[1] - A[1]) - (C[0] - A[0]) * (B[1] - A[1]), flip = (sa < 0) === up + const P = (q: [number, number]) => [q[0], q[1], z] + flip ? tri(P(A), P(C), P(B), [0, 0, up ? 1 : -1]) : tri(P(A), P(B), P(C), [0, 0, up ? 1 : -1]) + } + } + flat([[-hx, -hy], [hx, -hy], [hx, hy], [-hx, hy]], pockets.map((q) => q.pts), zTop, true) // верх с отверстиями + for (const pk of pockets) flat(pk.pts, [], zFloor, true) // дно карманов + flat([[-hx, -hy], [hx, -hy], [hx, hy], [-hx, hy]], [], 0, false) // дно корпуса + const quad = (a: number[], b: number[], c: number[], d: number[], na: number[], nb: number[]) => { // a,b снизу; c,d сверху (c над b, d над a) + pos.push(...a, ...b, ...d, ...b, ...c, ...d); nor.push(...na, ...nb, ...na, ...nb, ...nb, ...na) + } + for (const pk of pockets) { // стенки кармана: нормали в сторону кармана (аналитические или усреднённые) + const n = pk.pts.length, vn = pk.nrm.map((q) => [-q[0], -q[1], 0]) + for (let i = 0; i < n; i++) { const a = pk.pts[i], b = pk.pts[(i + 1) % n]; quad([a[0], a[1], zFloor], [b[0], b[1], zFloor], [b[0], b[1], zTop], [a[0], a[1], zTop], vn[i], vn[(i + 1) % n]) } + } + const sideN = [[0, -1, 0], [1, 0, 0], [0, 1, 0], [-1, 0, 0]], corners: [number, number][] = [[-hx, -hy], [hx, -hy], [hx, hy], [-hx, hy]] + for (let i = 0; i < 4; i++) { const a = corners[i], b = corners[(i + 1) % 4]; quad([b[0], b[1], 0], [a[0], a[1], 0], [a[0], a[1], zTop], [b[0], b[1], zTop], sideN[i], sideN[i]) } + const geo = new THREE.BufferGeometry(); geo.setAttribute('position', new THREE.BufferAttribute(new Float32Array(pos), 3)); geo.setAttribute('normal', new THREE.BufferAttribute(new Float32Array(nor), 3)) + mat.wireframe = chk('wire'); group.add(new THREE.Mesh(geo, mat)) + if (chk('tools')) for (const o of outs) { // инструменты: настоящий контур без зазора, стоят на дне кармана + const sh = new THREE.Shape(); const loop = o.loops.reduce((best, l) => (l.length > best.length ? l : best), o.loops[0]) + loop.forEach(([x, y], t) => { const X = x / sc - cx, Y = -(y / sc - cy); t ? sh.lineTo(X, Y) : sh.moveTo(X, Y) }) + const g2 = new THREE.ExtrudeGeometry(sh, { depth: Math.max(0.5, num('depth') - 0.6), bevelEnabled: false }); g2.translate(0, 0, zFloor + 0.05) + group.add(new THREE.Mesh(g2, toolMat)) + } + $('stat3d').textContent = `${pockets.length} кармана; ${chk('fit') ? `кривых Безье ${fitInfo.segs} (контрольных точек ${fitInfo.ctrl})` : `полилиния ${fitInfo.segs} вершин`}; вершин в модели ${pockets.reduce((a, p) => a + p.pts.length, 0)}, ${(pos.length / 9 / 1000).toFixed(0)} тыс. треуг.; глубина ${(zTop - zFloor).toFixed(0)} мм, зазор ${clearMm} мм` + const sig = `${binW}|${binH}`; if (sig !== fitted) { fitted = sig; frame(Math.max(binW, binH), 'iso') } +} + +// ---------- запуск модели и события ---------- +/** Второй проход: каждый найденный объект (рамка + поле) прогоняется моделью отдельно, поэтому на объект приходятся все 512 px сети (≈0.3 мм вместо ≈0.6 мм на весь лист). */ +async function refine(c: (typeof CANDIDATES)[number], ep: string, base: Float32Array, r: Rectified): Promise { + const seg0 = aiSegmentation(r, base), W = r.image.width, H = r.image.height, out = Float32Array.from(base), pad = Math.round(12 * r.scale) + for (const s of seg0.sils) { + const x0 = Math.max(0, s.bbox[0] - pad), y0 = Math.max(0, s.bbox[1] - pad), x1 = Math.min(W - 1, s.bbox[2] + pad), y1 = Math.min(H - 1, s.bbox[3] + pad), cw = x1 - x0 + 1, ch = y1 - y0 + 1 + const crop = new Uint8Array(cw * ch * 3); for (let j = 0; j < ch; j++) for (let i = 0; i < cw; i++) { const q = ((y0 + j) * W + x0 + i) * 3; crop.set(r.rgb!.subarray(q, q + 3), (j * cw + i) * 3) } + const res = await infer(c, ep, 1, crop, cw, ch, () => {}) + log(` уточнение объекта №${s.id}: ${cw}×${ch} px, ${res.ms.toFixed(0)} мс`) + for (let j = 0; j < ch; j++) for (let i = 0; i < cw; i++) { // внутри рамки без самого края (край рамки остаётся от общего прохода) + const edge = Math.min(i, j, cw - 1 - i, ch - 1 - j), w2 = Math.min(1, edge / (pad * 0.5)) // плавный переход к общему проходу на краю рамки + out[(y0 + j) * W + x0 + i] = base[(y0 + j) * W + x0 + i] * (1 - w2) + res.plane[j * cw + i] * w2 + } + } + return out +} + +$('runai').addEventListener('click', async () => { + if (!rect) return + const c = CANDIDATES.find((x) => x.id === $('modelsel').value)!, ep = $('ep').value, btn = $('runai'); btn.disabled = true + try { + log(`силуэты: ${c.id}, ${ep}, лицензия ${c.license}`) + const r = await infer(c, ep, 1, rect.rgb!, rect.image.width, rect.image.height, log) + log(`общий проход за ${r.ms.toFixed(0)} мс`); matte = r.plane + if (chk('refine')) { const t = performance.now(); matte = await refine(c, ep, matte, rect); log(`второй проход по объектам за ${((performance.now() - t) / 1000).toFixed(1)} с`) } + $('eng').value = 'ai'; schedule() + } catch (e) { log('ОШИБКА: ' + ((e as Error)?.message ?? e)) } finally { btn.disabled = false } +}) +$('sample').addEventListener('change', () => { $('file').value = ''; loadPhoto() }) +$('file').addEventListener('change', () => loadPhoto()) +for (const id of ['ppaper', 'porient']) $(id).addEventListener('change', () => { rebuildRect(); schedule() }) +for (const id of ['margin', 'sens']) $(id).addEventListener('change', () => { rebuildRect(); schedule() }) +for (const el of document.querySelectorAll('input[type=range], input[type=number], input[type=checkbox], select')) { + if (['file', 'sample', 'ppaper', 'porient', 'margin', 'sens', 'modelsel', 'ep'].includes(el.id) || el.closest('#sils')) continue + el.addEventListener('input', () => { if (el.id === 'lightang') setLight(); schedule() }) +} +loadPhoto() diff --git a/poc/ai/engine.ts b/poc/ai/engine.ts new file mode 100644 index 0000000..62a45c2 --- /dev/null +++ b/poc/ai/engine.ts @@ -0,0 +1,94 @@ +/** Запуск моделей PoC: загрузка (Cache API), сессии ORT Web, подготовка входа и разворот выхода. Не зависит от страницы. */ +import * as ort from 'onnxruntime-web' +import type { Candidate } from './models' + +ort.env.wasm.proxy = true // вывод в воркере ORT: страница не замирает +ort.env.wasm.wasmPaths = 'https://cdn.jsdelivr.net/npm/onnxruntime-web@' + (ort.env.versions?.web ?? 'latest') + '/dist/' + +const sessions = new Map() +export type Log = (s: string) => void + +async function fetchUrl(url: string, bytes: number, log: Log): Promise { + const cache = await caches.open('poc-ai-models') + let r = await cache.match(url) + const t0 = performance.now() + if (!r) { + const net = await fetch(url) + if (!net.ok) throw new Error(`HTTP ${net.status} for ${url}`) + await cache.put(url, net.clone()); r = net + log(`скачано ${(bytes / 1e6).toFixed(0)} МБ за ${((performance.now() - t0) / 1000).toFixed(1)} с`) + } + const buf = await r.arrayBuffer() + if (buf.byteLength !== bytes) log(`ВНИМАНИЕ: размер ${buf.byteLength} ≠ ${bytes}`) + return buf +} + +async function session(c: Candidate, ep: string, threads: number, log: Log): Promise { + const key = `${c.id}|${ep}|${threads}` + let s = sessions.get(key) + if (s) return s + const buf = await fetchUrl(c.url, c.bytes, log) + const externalData = c.data ? [{ path: c.data.path, data: new Uint8Array(await fetchUrl(c.data.url, c.data.bytes, log)) }] : undefined + ort.env.wasm.numThreads = threads + const t = performance.now() + s = await ort.InferenceSession.create(buf, { executionProviders: [ep], graphOptimizationLevel: 'all', ...(externalData ? { externalData } : {}) }) + log(`сессия ${c.id} (${ep}) за ${(performance.now() - t).toFixed(0)} мс`) + sessions.set(key, s) + return s +} + +/** Вход NCHW float32: картинка вписывается в s × s с сохранением пропорций, поля — среднее модели. */ +function toInput(rgb: Uint8Array, w: number, h: number, c: Candidate, s: number): Float32Array { + const src = new ImageData(w, h) + for (let i = 0; i < w * h; i++) { src.data[i * 4] = rgb[i * 3]; src.data[i * 4 + 1] = rgb[i * 3 + 1]; src.data[i * 4 + 2] = rgb[i * 3 + 2]; src.data[i * 4 + 3] = 255 } + const a = document.createElement('canvas'); a.width = w; a.height = h; a.getContext('2d')!.putImageData(src, 0, 0) + const b = document.createElement('canvas'); b.width = s; b.height = s + const g = b.getContext('2d')!; g.imageSmoothingQuality = 'high' + g.fillStyle = `rgb(${c.mean.map((m) => Math.round(m * 255)).join(',')})`; g.fillRect(0, 0, s, s) + const r = Math.min(s / w, s / h); g.drawImage(a, (s - w * r) / 2, (s - h * r) / 2, w * r, h * r) + const d = g.getImageData(0, 0, s, s).data + const out = new Float32Array(3 * s * s) + for (let i = 0; i < s * s; i++) for (let k = 0; k < 3; k++) out[k * s * s + i] = (d[i * 4 + k] / 255 - c.mean[k]) / c.std[k] + return out +} + +/** Бикубическая (Катмулл–Ром) интерполяция плоскости: гладкая изолиния при растяжении, без «ступеней» билинейной сетки. */ +export function cubicSample(v: Float32Array, w: number, h: number, x: number, y: number): number { + x -= 0.5; y -= 0.5 + const x0 = Math.floor(x), y0 = Math.floor(y), tx = x - x0, ty = y - y0 + const wgt = (t: number) => [(-t * t * t + 2 * t * t - t) / 2, (3 * t * t * t - 5 * t * t + 2) / 2, (-3 * t * t * t + 4 * t * t + t) / 2, (t * t * t - t * t) / 2] + const wx = wgt(tx), wy = wgt(ty); let acc = 0 + for (let j = 0; j < 4; j++) { + const yy = Math.min(h - 1, Math.max(0, y0 - 1 + j)); let row = 0 + for (let i = 0; i < 4; i++) row += wx[i] * v[yy * w + Math.min(w - 1, Math.max(0, x0 - 1 + i))] + acc += wy[j] * row + } + return acc +} + +/** Выход s × s → плоскость w × h (бикубически, поля вписывания отброшены), значения 0…1. */ +function fromOutput(v: Float32Array, s: number, w: number, h: number): Float32Array { + const out = new Float32Array(w * h), r = Math.min(s / w, s / h), ox = (s - w * r) / 2, oy = (s - h * r) / 2 + for (let j = 0; j < h; j++) for (let i = 0; i < w; i++) out[j * w + i] = Math.min(1, Math.max(0, cubicSample(v, s, s, (i + 0.5) * r + ox, (j + 0.5) * r + oy))) + return out +} + +/** Прогон модели на картинке rgb (w × h). matte → альфа 0…1, depth → глубина 0…1 по 2–98 перцентилям (ближе = больше). */ +export async function infer(c: Candidate, ep: string, threads: number, rgb: Uint8Array, w: number, h: number, log: Log): Promise<{ plane: Float32Array; ms: number }> { + const sess = await session(c, ep, threads, log) + const s = c.size, data = toInput(rgb, w, h, c, s) + const t = performance.now() + const out = await sess.run({ [sess.inputNames[0]]: new ort.Tensor('float32', data, [1, 3, s, s]) }) + const ms = performance.now() - t + let o = out[sess.outputNames[0]].data as Float32Array + if (c.kind === 'depth') { + const sorted = Float32Array.from(o).sort(), lo = sorted[Math.floor(sorted.length * 0.02)], hi = sorted[Math.floor(sorted.length * 0.98)] + o = o.map((x) => Math.min(1, Math.max(0, (x - lo) / (hi - lo || 1)))) + } else { + let lo = Infinity, hi = -Infinity; for (const x of o) { if (x < lo) lo = x; if (x > hi) hi = x } + if (c.sigmoid || lo < -0.001 || hi > 1.001) o = o.map((x) => 1 / (1 + Math.exp(-x))) + } + // выход может быть не s × s (некоторые модели): определяем сторону по длине + const side = Math.round(Math.sqrt(o.length)) + return { plane: fromOutput(o, side, w, h), ms } +} diff --git a/poc/ai/findings/cat_face_4_levels.jpg b/poc/ai/findings/cat_face_4_levels.jpg new file mode 100644 index 0000000..57c1608 Binary files /dev/null and b/poc/ai/findings/cat_face_4_levels.jpg differ diff --git a/poc/ai/findings/cat_face_local_contrast.jpg b/poc/ai/findings/cat_face_local_contrast.jpg new file mode 100644 index 0000000..85b3a30 Binary files /dev/null and b/poc/ai/findings/cat_face_local_contrast.jpg differ diff --git a/poc/ai/findings/cat_whole_local_contrast.jpg b/poc/ai/findings/cat_whole_local_contrast.jpg new file mode 100644 index 0000000..5327763 Binary files /dev/null and b/poc/ai/findings/cat_whole_local_contrast.jpg differ diff --git a/poc/ai/index.html b/poc/ai/index.html new file mode 100644 index 0000000..240c385 --- /dev/null +++ b/poc/ai/index.html @@ -0,0 +1,117 @@ + +PoC: Keychain — пайплайн фото → рельеф + +
PoC Keychain: фото → рельеф, пайплайн по этапам · ← PoC фото-вырезов (change 32) + · isolated: , cores:
+
+
+

1Изображение и кроп

+ + +
+ + + + + +
+ + + +
+

2Силуэт

+ +
+ + +
+
+ +
+ + + + + + + + + + +
+

3Глубина

+ + + + + + +
Как применяется: сырая глубина → (привязка к краям фото, чтобы не было зернистости) → нормировка по 2–98 % внутри силуэта (ближе = выше) → гамма → сглаживание → доля «глубина» в смеси. Чтобы увидеть только её, выберите «только глубина» под 3D.
+ +
+

4Детали (локальный контраст)

+ + + + + + + + +
+

5Смесь формы → высота

+
+ +
+
пресет:
+ +
+ +
+ + + 1.5 ≈ 56° от горизонтали: отвесные ступени становятся склонами + +
+
+ +
+ + diff --git a/poc/ai/keychain.ts b/poc/ai/keychain.ts new file mode 100644 index 0000000..1d4f321 --- /dev/null +++ b/poc/ai/keychain.ts @@ -0,0 +1,423 @@ +/** + * PoC пайплайна Keychain (change 33 + 34): фото → кроп → силуэт (модель маски) → глубина (модель глубины) → детали + * (локальный контраст) → смесь → 3D-рельеф. Этапы независимы: у каждого своя модель/настройки и своё превью. + */ +import * as THREE from 'three' +import { OrbitControls } from 'three/addons/controls/OrbitControls.js' +import { edt } from '../../src/geometry/raster/edt' +import { gaussianBlur, MIN_SIGMA } from '../../src/geometry/raster/blur' +import { otsu } from '../../src/geometry/raster/threshold' +import { close } from '../../src/geometry/raster/morphology' +import { findHoles, fillHoles, labelComponents, removeSpecks } from '../../src/geometry/raster/components' +import { backendsOf, CANDIDATES, type Candidate } from './models' +import { infer } from './engine' +import { findCorners, fitLoop, signedDistance, tessellate, vectorOutline, type Pt } from './outline' + +const $ = (id: string) => document.getElementById(id) as T +const num = (id: string) => +$(id).value +const chk = (id: string) => $(id).checked +const logEl = $('log') +const log = (s: string) => { logEl.textContent += s + '\n'; logEl.scrollTop = logEl.scrollHeight; console.log(s) } +$('coi').textContent = String(self.crossOriginIsolated); $('cores').textContent = String(navigator.hardwareConcurrency) + +for (const c of CANDIDATES.filter((x) => (x.kind ?? 'matte') === 'matte')) $('silmodelsel').append(new Option(c.label, c.id)) +for (const c of CANDIDATES.filter((x) => x.kind === 'depth')) $('depthsel').append(new Option(c.label, c.id)) +$('silmodelsel').value = 'birefnet-lite-512-fp16' +/** Бэкенд подбирается под модель: показываются только те, на которых она работает. */ +function syncBackends(modelSel: string, epSel: string) { + const ep = $(epSel), list = backendsOf($(modelSel).value), keep = ep.value + ep.innerHTML = ''; for (const b of list) ep.append(new Option(b, b)) + ep.value = list.includes(keep as 'wasm' | 'webgpu') ? keep : list[0]; ep.disabled = list.length === 1 +} +for (const [m, e] of [['silmodelsel', 'silep'], ['depthsel', 'depthep']] as const) { syncBackends(m, e); $(m).addEventListener('change', () => syncBackends(m, e)) } + +// ---------- состояние ---------- +interface Img { w: number; h: number; rgb: Uint8Array } +interface Crop { x: number; y: number; w: number; h: number } // доли +let img: Img | null = null +let matte: Float32Array | null = null // мягкая альфа, размер картинки +let depthRec: { plane: Float32Array; w: number; h: number; crop: Crop } | null = null + +const cropNow = (): Crop => ({ x: num('cx') / 100, y: num('cy') / 100, w: Math.min(num('cw'), 100 - num('cx')) / 100, h: Math.min(num('ch'), 100 - num('cy')) / 100 }) +const cropPx = (c: Crop, im: Img) => ({ x0: Math.round(c.x * im.w), y0: Math.round(c.y * im.h), pw: Math.max(8, Math.round(c.w * im.w)), ph: Math.max(8, Math.round(c.h * im.h)) }) + +async function loadImage() { + const f = $('file').files?.[0] + const el = new Image(); el.src = f ? URL.createObjectURL(f) : `./samples/${$('sample').value}.jpeg`; await el.decode() + const k = Math.min(1, 2048 / Math.max(el.naturalWidth, el.naturalHeight)) + const cv = document.createElement('canvas'); cv.width = Math.round(el.naturalWidth * k); cv.height = Math.round(el.naturalHeight * k) + const g = cv.getContext('2d')!; g.drawImage(el, 0, 0, cv.width, cv.height) + const d = g.getImageData(0, 0, cv.width, cv.height).data, rgb = new Uint8Array(cv.width * cv.height * 3) + for (let i = 0; i < cv.width * cv.height; i++) { rgb[i * 3] = d[i * 4]; rgb[i * 3 + 1] = d[i * 4 + 1]; rgb[i * 3 + 2] = d[i * 4 + 2] } + img = { w: cv.width, h: cv.height, rgb }; matte = null; depthRec = null; work = null + log(`снимок ${img.w}×${img.h}`); schedule() +} + +// ---------- рабочий растр кропа ---------- +interface Work { key: string; ww: number; wh: number; gray: Float32Array; rgb: Uint8Array; matte: Float32Array | null; depth: Float32Array | null; k: number; x0: number; y0: number } +let work: Work | null = null +function bil(p: Float32Array, w: number, h: number, x: number, y: number): number { + x = Math.min(w - 1.001, Math.max(0, x - 0.5)); y = Math.min(h - 1.001, Math.max(0, y - 0.5)) + const x0 = Math.floor(x), y0 = Math.floor(y), tx = x - x0, ty = y - y0 + return (p[y0 * w + x0] * (1 - tx) + p[y0 * w + x0 + 1] * tx) * (1 - ty) + (p[(y0 + 1) * w + x0] * (1 - tx) + p[(y0 + 1) * w + x0 + 1] * tx) * ty +} +function getWork(): Work | null { + if (!img) return null + const c = cropNow(), { x0, y0, pw, ph } = cropPx(c, img) + const key = [num('res'), x0, y0, pw, ph, matte ? 1 : 0, depthRec ? depthRec.crop.x + ',' + depthRec.crop.y + ',' + depthRec.crop.w + ',' + depthRec.crop.h : 0].join('|') + if (work && work.key === key) return work + const k = Math.min(1, num('res') / Math.max(pw, ph)), ww = Math.max(8, Math.round(pw * k)), wh = Math.max(8, Math.round(ph * k)) + const gray = new Float32Array(ww * wh), rgb = new Uint8Array(ww * wh * 3) + const mw = matte ? new Float32Array(ww * wh) : null, dw = depthRec ? new Float32Array(ww * wh) : null + const dpx = depthRec ? cropPx(depthRec.crop, img) : null + for (let j = 0; j < wh; j++) for (let i = 0; i < ww; i++) { + const fx = x0 + (i + 0.5) / k, fy = y0 + (j + 0.5) / k // координаты в полном кадре + const ix = Math.min(img.w - 1, Math.floor(fx)), iy = Math.min(img.h - 1, Math.floor(fy)), q = (iy * img.w + ix) * 3, p = j * ww + i + rgb[p * 3] = img.rgb[q]; rgb[p * 3 + 1] = img.rgb[q + 1]; rgb[p * 3 + 2] = img.rgb[q + 2] + gray[p] = (0.299 * img.rgb[q] + 0.587 * img.rgb[q + 1] + 0.114 * img.rgb[q + 2]) / 255 + if (mw && matte) mw[p] = bil(matte, img.w, img.h, fx, fy) + if (dw && depthRec && dpx) dw[p] = bil(depthRec.plane, depthRec.w, depthRec.h, (fx - dpx.x0) * depthRec.w / dpx.pw, (fy - dpx.y0) * depthRec.h / dpx.ph) + } + work = { key, ww, wh, gray, rgb, matte: mw, depth: dw, k, x0, y0 } + return work +} + +// ---------- этапы ---------- +const ones = (n: number) => new Uint8Array(n).fill(1) +const smooth01 = (t: number) => { const x = Math.min(1, Math.max(0, t)); return x * x * (3 - 2 * x) } + +/** Гауссово сглаживание только внутри маски (нормированная свёртка): значения снаружи не просачиваются внутрь. */ +function blurIn(a: Float32Array, w: number, h: number, sigmaPx: number, mask: Uint8Array): Float32Array { + return sigmaPx < 0.4 ? a : gaussianBlur(a, w, h, Math.max(MIN_SIGMA, sigmaPx), mask) +} +/** Предел наклона: нижняя и верхняя огибающие конусом (две хамфер-проходки), результат — среднее. Отвесные ступени становятся склонами. */ +function limitSlope(v: Float32Array, w: number, h: number, L: number, mask: Uint8Array): Float32Array { + if (L <= 0) return v + const lo = Float32Array.from(v), hi = Float32Array.from(v), D = L * Math.SQRT2 + const pass = (fwd: boolean) => { + const y0 = fwd ? 0 : h - 1, y1 = fwd ? h : -1, dy = fwd ? 1 : -1, x0 = fwd ? 0 : w - 1, x1 = fwd ? w : -1, dx = fwd ? 1 : -1 + for (let y = y0; y !== y1; y += dy) for (let x = x0; x !== x1; x += dx) { + const i = y * w + x; if (!mask[i]) continue + for (const [ox, oy, d] of [[-dx, 0, L], [0, -dy, L], [-dx, -dy, D], [dx, -dy, D]] as const) { + const xx = x + ox, yy = y + oy; if (xx < 0 || yy < 0 || xx >= w || yy >= h) continue + const j = yy * w + xx; if (!mask[j]) continue + if (lo[j] + d < lo[i]) lo[i] = lo[j] + d + if (hi[j] - d > hi[i]) hi[i] = hi[j] - d + } + } + } + pass(true); pass(false) + return v.map((_, i) => 0.5 * (lo[i] + hi[i])) +} + +/** Среднее по окну (2r+1)² с зажимом у краёв, бегущими суммами. */ +function box(a: Float32Array, w: number, h: number, r: number): Float32Array { + const tmp = new Float64Array(w * h), out = new Float32Array(w * h) + for (let y = 0; y < h; y++) { + let acc = 0; const row = y * w + for (let x = -r; x <= r; x++) acc += a[row + Math.min(w - 1, Math.max(0, x))] + for (let x = 0; x < w; x++) { tmp[row + x] = acc; acc += a[row + Math.min(w - 1, x + r + 1)] - a[row + Math.max(0, x - r)] } + } + const n = (2 * r + 1) * (2 * r + 1) + for (let x = 0; x < w; x++) { + let acc = 0 + for (let y = -r; y <= r; y++) acc += tmp[Math.min(h - 1, Math.max(0, y)) * w + x] + for (let y = 0; y < h; y++) { out[y * w + x] = acc / n; acc += tmp[Math.min(h - 1, y + r + 1) * w + x] - tmp[Math.max(0, y - r) * w + x] } + } + return out +} +/** Направляющий фильтр (He): сглаживает p, но края берёт из guide (яркость фото). Убирает зернистость глубины, не размывая контуры черт. */ +function guided(guide: Float32Array, p: Float32Array, w: number, h: number, r: number, eps: number): Float32Array { + if (r < 1) return p + const mI = box(guide, w, h, r), mp = box(p, w, h, r) + const II = box(guide.map((v) => v * v), w, h, r), Ip = box(guide.map((v, i) => v * p[i]), w, h, r) + const a = new Float32Array(w * h), b = new Float32Array(w * h) + for (let i = 0; i < w * h; i++) { const v = II[i] - mI[i] * mI[i], c = Ip[i] - mI[i] * mp[i]; a[i] = c / (v + eps); b[i] = mp[i] - a[i] * mI[i] } + const ma = box(a, w, h, r), mb = box(b, w, h, r) + return guide.map((v, i) => ma[i] * v + mb[i]) +} + +function silhouette(w: Work, pxMm: number): { mask: Uint8Array; note: string } { + const n = w.ww * w.wh, src = $('silsrc').value + let mask: Uint8Array, note = '' + if (src === 'frame') mask = ones(n) + else if (src === 'brightness') { + const g8 = Uint8Array.from(w.gray, (v) => Math.round(v * 255)), thr = otsu(g8), inv = chk('inv') + mask = new Uint8Array(n); for (let i = 0; i < n; i++) mask[i] = (g8[i] < thr) !== inv ? 1 : 0 + mask = close(mask, w.ww, w.wh, Math.max(1, Math.round(0.6 * pxMm))) + note = `порог Оцу ${thr}` + } else { + if (!w.matte) { note = 'маска не посчитана: запустите модель'; return { mask: ones(n), note } } + const s = num('smooth') * pxMm, a = s > 0 ? gaussianBlur(w.matte, w.ww, w.wh, Math.max(MIN_SIGMA, s), ones(n)) : w.matte + mask = new Uint8Array(n); const thr = num('thr'); for (let i = 0; i < n; i++) mask[i] = a[i] >= thr ? 1 : 0 + if (chk('largest')) { + const cc = labelComponents(mask, w.ww, w.wh); let best = 0 + for (let k = 1; k <= cc.count; k++) if (cc.sizes[k] > (cc.sizes[best] ?? 0)) best = k + mask = mask.map((_, i) => (cc.labels[i] === best && best ? 1 : 0)) as Uint8Array + } + } + if (src === 'brightness') { const b = blurIn(Float32Array.from(mask), w.ww, w.wh, 0.6, ones(n)); mask = b.map((v) => (v >= 0.5 ? 1 : 0)) as unknown as Uint8Array } // ровный край без пиксельной лесенки + if (src !== 'frame') { + if (chk('fill')) { // мелкие области (между штрихами) заливаются, крупные (отверстие кольца) остаются открытыми + const open = findHoles(mask, w.ww, w.wh).holes.filter((x) => x.size / (pxMm * pxMm) >= num('openmm2')).map((x) => x.id) + mask = fillHoles(mask, w.ww, w.wh, open).mask + note += (note ? ', ' : '') + `открытых областей: ${open.length}` + } + mask = removeSpecks(mask, w.ww, w.wh, Math.round(2 * pxMm * pxMm)) + } + return { mask, note } +} + +function normDepth(w: Work, mask: Uint8Array, pxMm: number): Float32Array | null { + if (!w.depth) return null + const src = guided(w.gray, w.depth, w.ww, w.wh, Math.round(num('dgr') * pxMm), 1e-3) // привязка к краям фото до нормировки + const v: number[] = []; for (let i = 0; i < mask.length; i++) if (mask[i]) v.push(src[i]) + if (!v.length) return null + v.sort((a, b) => a - b); const lo = v[Math.floor(v.length * 0.02)], hi = v[Math.floor(v.length * 0.98)], inv = chk('dinv') + return src.map((d) => { const t = Math.min(1, Math.max(0, (d - lo) / (hi - lo || 1))); return inv ? 1 - t : t }) +} + +function detail(w: Work, pxMm: number): Float32Array { + const amount = num('lca'), n = w.ww * w.wh + if (amount <= 0) return new Float32Array(n).fill(0.5) + const sg = Math.max(MIN_SIGMA, num('lcr') * pxMm), m = ones(n) + const src = num('tpre') > 0 ? gaussianBlur(w.gray, w.ww, w.wh, Math.max(MIN_SIGMA, num('tpre') * pxMm), m) : w.gray // шум и зерно JPEG убираются до усиления + const mean = gaussianBlur(src, w.ww, w.wh, sg, m), dev = src.map((v, i) => v - mean[i]) + const sd = gaussianBlur(dev.map((v) => v * v), w.ww, w.wh, sg, m).map((v) => Math.sqrt(v) + 0.02) + let t = dev.map((v, i) => 0.5 + 0.5 * Math.tanh(amount * v / sd[i])) + const L = num('lvl'); if (L >= 2) t = t.map((v) => Math.round(v * (L - 1)) / (L - 1)) + return t +} + +// ---------- превью ---------- +function show(id: string, w: number, h: number, px: (i: number) => [number, number, number]) { + const cv = $(id); cv.width = w; cv.height = h + const g = cv.getContext('2d')!, im = g.createImageData(w, h) + for (let i = 0; i < w * h; i++) { const [r, gg, b] = px(i); im.data[i * 4] = r; im.data[i * 4 + 1] = gg; im.data[i * 4 + 2] = b; im.data[i * 4 + 3] = 255 } + g.putImageData(im, 0, 0) +} +/** Подложка-шашка (как «прозрачность» в графических редакторах) для всего, что вне силуэта: клетка 8 px. */ +const checker = (i: number, w: number): [number, number, number] => { const x = (i % w) >> 3, y = Math.floor(i / w) >> 3; return (x + y) & 1 ? [214, 214, 214] : [170, 170, 170] } +const gr = (v: number): [number, number, number] => { const t = Math.round(Math.min(1, Math.max(0, v)) * 255); return [t, t, t] } + +function previewImage(c: Crop) { + if (!img) return + const cv = $('pv-img'), s = Math.min(1, 520 / img.w); cv.width = Math.round(img.w * s); cv.height = Math.round(img.h * s) + const g = cv.getContext('2d')!, im = g.createImageData(img.w, img.h) + for (let i = 0; i < img.w * img.h; i++) { im.data[i * 4] = img.rgb[i * 3]; im.data[i * 4 + 1] = img.rgb[i * 3 + 1]; im.data[i * 4 + 2] = img.rgb[i * 3 + 2]; im.data[i * 4 + 3] = 255 } + const t = document.createElement('canvas'); t.width = img.w; t.height = img.h; t.getContext('2d')!.putImageData(im, 0, 0) + g.drawImage(t, 0, 0, cv.width, cv.height) + g.fillStyle = 'rgba(0,0,0,0.45)'; g.beginPath(); g.rect(0, 0, cv.width, cv.height); g.rect(c.x * cv.width, c.y * cv.height, c.w * cv.width, c.h * cv.height); g.fill('evenodd') + g.strokeStyle = '#ff3'; g.lineWidth = 2; g.strokeRect(c.x * cv.width, c.y * cv.height, c.w * cv.width, c.h * cv.height) +} + +// ---------- обновление пайплайна ---------- +let timer = 0 +function schedule() { clearTimeout(timer); timer = window.setTimeout(update, 120) } +function update() { + for (const id of ['cx', 'cy', 'cw', 'ch', 'thr', 'smooth', 'wc', 'wd', 'wt', 'roll']) { const e = $(id); (e.nextElementSibling as HTMLElement).textContent = e.value } + $('silmodel').hidden = $('silsrc').value !== 'model' + $('silbright').hidden = $('silsrc').value !== 'brightness' + const c = cropNow(); previewImage(c) + const w = getWork(); if (!w || !img) return + const partW = num('partw'), pxMm = w.ww / partW, partH = partW * w.wh / w.ww, n = w.ww * w.wh + const sil = silhouette(w, pxMm), note = sil.note + const silMask = sil.mask // силуэт всегда показывается в превью этапа 2, даже когда слой «маска» выключен для 3D + const mask = chk('lm') ? silMask : ones(n) // слой «маска»: выключен — рельеф без вырезания по силуэту + $('st-sil').textContent = note + show('pv-mask', w.ww, w.wh, (i) => (silMask[i] ? [w.rgb[i * 3], w.rgb[i * 3 + 1], w.rgb[i * 3 + 2]] : checker(i, w.ww))) + let dn = normDepth(w, mask, pxMm) + if (dn) { const g = num('dgam'); if (g !== 1) dn = dn.map((v) => Math.pow(v, g)); dn = blurIn(dn, w.ww, w.wh, num('dsm') * pxMm, mask) } + const same = depthRec && JSON.stringify(depthRec.crop) === JSON.stringify(c) + $('st-depth').textContent = depthRec ? (same ? 'посчитана для этого кропа' : 'посчитана для другого кропа (можно пересчитать)') : 'не посчитана' + show('pv-depth', w.ww, w.wh, (i) => (mask[i] ? (dn ? gr(dn[i]) : [90, 90, 90]) : checker(i, w.ww))) + const tone = blurIn(detail(w, pxMm), w.ww, w.wh, num('tsm') * pxMm, mask) + // режим деталей: бугры (тон → высота), гравировка (только тёмные черты продавливаются канавками) или оба + const dmode = $('dmode').value, dth = Math.min(0.95, num('dth')) + const dt = tone.map((t) => { + const groove = smooth01((Math.max(0, 0.5 - t) * 2 - dth) / (1 - dth)) // 0 везде, кроме самых тёмных мест + return dmode === 'bumps' ? t : dmode === 'engrave' ? 1 - groove : 0.5 * t + 0.5 * (1 - groove) + }) + show('pv-detail', w.ww, w.wh, (i) => (mask[i] ? gr(dt[i]) : checker(i, w.ww))) + // смесь + const dist = edt(mask, w.ww, w.wh), bevelPx = Math.max(1, num('bevel') * pxMm), roll = num('roll') / 100 + const wc = chk('lc') ? num('wc') : 0, wd = dn && chk('ld') ? num('wd') : 0, wt = chk('lt') ? num('wt') : 0 + const solo = (document.querySelector('input[name=solo]:checked') as HTMLInputElement).value + const value = new Float32Array(n) + for (let i = 0; i < n; i++) { + if (!mask[i]) continue + const contour = smooth01(dist[i] / bevelPx), tot = wc + wd + wt, mixed = solo === 'contour' ? contour : solo === 'depth' ? (dn ? dn[i] : 0) : solo === 'detail' ? dt[i] : tot === 0 ? contour : (wc * contour + wd * (dn ? dn[i] : 0) + wt * dt[i]) / tot + value[i] = mixed * ((1 - roll) + roll * contour) + } + let final = blurIn(value, w.ww, w.wh, num('fsm') * pxMm, mask) + { const rng = Math.max(0.01, num('range')); final = limitSlope(final, w.ww, w.wh, num('slope') / pxMm / rng, mask) } // наклон: мм высоты на мм детали → доли диапазона на пиксель + value.set(final) + show('pv-mix', w.ww, w.wh, (i) => (mask[i] ? gr(value[i]) : checker(i, w.ww))) + build3D(w, mask, value, partW, partH) +} + +// ---------- 3D ---------- +const view = $('view') +const renderer = new THREE.WebGLRenderer({ antialias: true }); view.append(renderer.domElement) +const scene = new THREE.Scene(); scene.background = new THREE.Color(0xe8e8e4) +const camera = new THREE.PerspectiveCamera(35, 1, 1, 2000) +const controls = new OrbitControls(camera, renderer.domElement) +const hemi = new THREE.HemisphereLight(0xffffff, 0x666666, 0.55); scene.add(hemi) +const sun = new THREE.DirectionalLight(0xffffff, 2.2); scene.add(sun) +const fill = new THREE.DirectionalLight(0xffffff, 0.5); fill.position.set(60, -80, 40); scene.add(fill) +const mat = new THREE.MeshStandardMaterial({ color: 0xd8d2c4, roughness: 0.75, metalness: 0 }) +const group = new THREE.Group(); scene.add(group) +function setLight() { const a = (num('lightang') * Math.PI) / 180; sun.position.set(Math.cos(a) * 80, Math.sin(a) * 80, 70) } +function resize() { const w = view.clientWidth, h = view.clientHeight; renderer.setSize(w, h, false); camera.aspect = w / h; camera.updateProjectionMatrix() } +new ResizeObserver(resize).observe(view); resize(); setLight() +renderer.setAnimationLoop(() => { controls.update(); renderer.render(scene, camera) }) +let fitted = '' +function frame(partW: number, partH: number, preset: 'top' | 'iso' | null) { + const d = Math.max(partW, partH) * 1.9 + if (preset === 'top') camera.position.set(0, -0.001, d) + else if (preset === 'iso') camera.position.set(-d * 0.45, -d * 0.7, d * 0.75) + camera.up.set(0, 0, 1); if (preset === 'top') camera.up.set(0, 1, 0) + camera.lookAt(0, 0, 0); controls.target.set(0, 0, 0); controls.update() +} +$('vtop').addEventListener('click', () => frame(num('partw'), num('partw'), 'top')) +$('viso').addEventListener('click', () => frame(num('partw'), num('partw'), 'iso')) + +const log2 = document.createElement('span'); log2.className = 'note'; $('stat3d').after(log2) +function build3D(w: Work, mask: Uint8Array, value: Float32Array, partW: number, partH: number) { + const nodes = Math.max(60, num('nodes')), s = Math.max(1, Math.ceil(Math.max(w.ww, w.wh) / nodes)) + const nx = Math.floor((w.ww - 1) / s) + 1, ny = Math.floor((w.wh - 1) / s) + 1 + const floor = num('floor'), range = num('range'), mmx = partW / w.ww, mmy = partH / w.wh + // мягкая маска: размытие в мм (в вырезах σ = 1 px при 4 px/мм ≈ 0.25 мм); по изолинии 0.5 край кладётся с субпиксельной точностью + const soft = gaussianBlur(Float32Array.from(mask), w.ww, w.wh, Math.max(1, num('osm') * (w.ww / partW)), ones(w.ww * w.wh)) // как в вырезах: σ ≈ 0.25 мм + const px = (i: number) => Math.min(w.ww - 0.5, i * s + 0.5), py = (j: number) => Math.min(w.wh - 0.5, j * s + 0.5) + const F = new Float32Array(nx * ny), isIn = new Uint8Array(nx * ny), Z = new Float32Array(nx * ny) + // векторный контур: изолиния, сглаживание вдоль кривой, острые углы сохраняются; поле F у края заменяется знаковым расстоянием до кривой + let loops: ReturnType | null = null + let wallS: { x: number; y: number; nx: number; ny: number }[] = [] // выборка кривых (мм детали) с нормалями наружу: для гладких стенок + if (chk('vec')) { + const f1 = gaussianBlur(Float32Array.from(mask), w.ww, w.wh, 0.8, ones(w.ww * w.wh)), pxMmL = w.ww / partW + loops = vectorOutline(Uint8Array.from(f1, (v) => Math.round(v * 255)), w.ww, w.wh, Math.max(0.5, num('osm') * pxMmL), num('cang')) + log2.textContent = `контур: ${loops.length} петель, ${loops.reduce((a, l) => a + l.length, 0)} вершин` + if (chk('fit')) { // подгонка кубических Безье (как в вырезах): минимум точек, аналитические нормали стенки + const tolPx = num('ftol') * pxMmL, stepPx = Math.max(1, num('fstep') * pxMmL); let segs = 0, ctrl = 0, verts = 0 + const fitted: Pt[][] = []; wallS = [] + for (const l of loops) { + if (l.length < 12) { fitted.push(l); continue } + const { segs: sg } = fitLoop(l, findCorners(l, 3, num('cang')), tolPx), sm = tessellate(sg, stepPx) + // направление нормали: наружу — туда, где маска пуста + let inside = 0; for (const q of sm) { const x = q.p[0] + q.n[0] * 1.5, y = q.p[1] + q.n[1] * 1.5; if (bil(soft, w.ww, w.wh, x, y) > 0.5) inside++ } + const flip = inside > sm.length / 2 ? -1 : 1 + fitted.push(sm.map((q) => q.p as Pt)); for (const q of sm) wallS.push({ x: q.p[0] * mmx - partW / 2, y: partH / 2 - q.p[1] * mmy, nx: q.n[0] * flip, ny: -q.n[1] * flip }) + segs += sg.length; ctrl += 3 * sg.length + 1; verts += sm.length + } + loops = fitted; log2.textContent = `контур: кривых Безье ${segs} (контрольных точек ${ctrl}), вершин в модели ${verts}, допуск ${num('ftol')} мм, шаг ${num('fstep')} мм` + } + } else log2.textContent = '' + for (let j = 0; j < ny; j++) for (let i = 0; i < nx; i++) { + const k = j * nx + i; F[k] = bil(soft, w.ww, w.wh, px(i), py(j)) + if (loops && F[k] > 0.003 && F[k] < 0.997) F[k] = Math.min(1, Math.max(0, 0.5 + signedDistance(loops, px(i), py(j)) / (2 * s))) // у края: расстояние до кривой в долях ячейки + isIn[k] = F[k] >= 0.5 ? 1 : 0 + Z[k] = floor + range * bil(value, w.ww, w.wh, px(i), py(j)) + } + const NX = new Float32Array(nx * ny), NY = new Float32Array(nx * ny) + for (let j = 0; j < ny; j++) for (let i = 0; i < nx; i++) { const k = j * nx + i; NX[k] = px(i) * mmx - partW / 2; NY[k] = partH / 2 - py(j) * mmy } + // Край: marching squares. Ячейка режется по линии F = 0.5: вершины контура лежат на изолинии мягкой маски, а не на узлах сетки. + const vx: number[] = [], vy: number[] = [], vz: number[] = [], vmap = new Map() + const corner = (k: number) => { let id = vmap.get(k); if (id === undefined) { id = vx.length; vmap.set(k, id); vx.push(NX[k]); vy.push(NY[k]); vz.push(Z[k]) } return id } + const cross = (a: number, b: number) => { // a внутри, b снаружи (или наоборот): точка F = 0.5 на ребре, высота — как у внутреннего узла + const key = -1 - (Math.min(a, b) * nx * ny + Math.max(a, b)); let id = vmap.get(key) + if (id === undefined) { + const ia = isIn[a] === 1, inn = ia ? a : b, out = ia ? b : a, t = (F[inn] - 0.5) / Math.max(1e-6, F[inn] - F[out]) + id = vx.length; vmap.set(key, id); vx.push(NX[inn] + (NX[out] - NX[inn]) * t); vy.push(NY[inn] + (NY[out] - NY[inn]) * t); vz.push(Z[inn]) + } + return id + } + const idx: number[] = [], segs: [number, number][] = [], flat: [number, number, number][] = [] // flat — ячейки для дна + for (let j = 0; j < ny - 1; j++) for (let i = 0; i < nx - 1; i++) { + const k0 = j * nx + i, ks = [k0, k0 + 1, k0 + nx + 1, k0 + nx] // по кругу + const n = ks.filter((k) => isIn[k]).length; if (!n) continue + const poly: number[] = [], crossIdx: boolean[] = [] + for (let e = 0; e < 4; e++) { + const a = ks[e], b = ks[(e + 1) % 4] + if (isIn[a]) { poly.push(corner(a)); crossIdx.push(false) } + if (isIn[a] !== isIn[b]) { poly.push(cross(a, b)); crossIdx.push(true) } + } + for (let t = 1; t + 1 < poly.length; t++) idx.push(poly[0], poly[t], poly[t + 1]) + for (let t = 0; t < poly.length; t++) { const u = (t + 1) % poly.length; if (crossIdx[t] && crossIdx[u]) segs.push([poly[t], poly[u]]) } + void flat + } + const pos = new Float32Array(vx.length * 3) + for (let v = 0; v < vx.length; v++) { pos[v * 3] = vx[v]; pos[v * 3 + 1] = vy[v]; pos[v * 3 + 2] = vz[v] } + const topGeo = new THREE.BufferGeometry(); topGeo.setAttribute('position', new THREE.BufferAttribute(pos, 3)); topGeo.setIndex(idx); topGeo.computeVertexNormals() + // Стенки по сегментам контура и дно: те же полигоны на z = 0 (зеркально) + const wp: number[] = [], wn: number[] = [] + const V = (v: number, z?: number): number[] => [vx[v], vy[v], z ?? vz[v]] + const nrmAt = new Map() + const wallN = (v: number): [number, number] => { // нормаль стенки в вершине контура: у ближайшей точки подогнанной кривой + let r = nrmAt.get(v); if (r) return r + let best = Infinity; r = [0, 0] + for (const q of wallS) { const d = (q.x - vx[v]) ** 2 + (q.y - vy[v]) ** 2; if (d < best) { best = d; r = [q.nx, q.ny] } } + nrmAt.set(v, r); return r + } + const tri3 = (a: number[], b: number[], c: number[], na: number[], nb: number[], nc: number[]) => { // порядок вершин так, чтобы лицевая сторона смотрела по нормали + const g = [(b[1] - a[1]) * (c[2] - a[2]) - (b[2] - a[2]) * (c[1] - a[1]), (b[2] - a[2]) * (c[0] - a[0]) - (b[0] - a[0]) * (c[2] - a[2]), (b[0] - a[0]) * (c[1] - a[1]) - (b[1] - a[1]) * (c[0] - a[0])] + const dn = g[0] * (na[0] + nb[0] + nc[0]) + g[1] * (na[1] + nb[1] + nc[1]) + g[2] * (na[2] + nb[2] + nc[2]) + if (dn >= 0) { wp.push(...a, ...b, ...c); wn.push(...na, ...nb, ...nc) } else { wp.push(...a, ...c, ...b); wn.push(...na, ...nc, ...nb) } + } + const curved = chk('vec') && chk('fit') && wallS.length > 0 + for (const [p0, p1] of segs) { + const n0 = curved ? [...wallN(p0), 0] : [0, 0, 0], n1 = curved ? [...wallN(p1), 0] : [0, 0, 0] + if (curved) { tri3(V(p0, 0), V(p1, 0), V(p0), n0, n1, n0); tri3(V(p1, 0), V(p1), V(p0), n1, n1, n0) } + else wp.push(...V(p0, 0), ...V(p1, 0), ...V(p0), ...V(p1, 0), ...V(p1), ...V(p0)) + } + for (let t = 0; t < idx.length; t += 3) { wp.push(...V(idx[t], 0), ...V(idx[t + 2], 0), ...V(idx[t + 1], 0)); if (curved) wn.push(0, 0, -1, 0, 0, -1, 0, 0, -1) } // дно + const skirt = new THREE.BufferGeometry(); skirt.setAttribute('position', new THREE.BufferAttribute(new Float32Array(wp), 3)) + if (curved) skirt.setAttribute('normal', new THREE.BufferAttribute(new Float32Array(wn), 3)); else skirt.computeVertexNormals() + group.clear() + mat.wireframe = chk('wire'); mat.side = THREE.DoubleSide + group.add(new THREE.Mesh(topGeo, mat), new THREE.Mesh(skirt, mat)) + $('stat3d').textContent = `сетка ${nx}×${ny}, шаг ${(s * mmx).toFixed(2)} мм, ${(idx.length / 3 / 1000).toFixed(0)} тыс. треуг.; деталь ${partW.toFixed(0)}×${partH.toFixed(0)} мм, высота до ${(floor + range).toFixed(1)} мм` + const sig = `${partW}|${partH}`; if (sig !== fitted) { fitted = sig; frame(partW, partH, 'iso') } +} + +// ---------- запуск моделей ---------- +async function runModel(kind: 'sil' | 'depth') { + if (!img) return + const c: Candidate = CANDIDATES.find((x) => x.id === $(kind === 'sil' ? 'silmodelsel' : 'depthsel').value)! + const ep = $(kind === 'sil' ? 'silep' : 'depthep').value + const btn = $(kind === 'sil' ? 'runsil' : 'rundepth'); btn.disabled = true + try { + log(`${kind === 'sil' ? 'силуэт' : 'глубина'}: ${c.id}, ${ep}, лицензия ${c.license}`) + let rgb = img.rgb, w = img.w, h = img.h, crop: Crop | null = null + if (kind === 'depth') { // глубина считается по кропу: на лице больше пикселей + crop = cropNow(); const p = cropPx(crop, img); w = p.pw; h = p.ph; rgb = new Uint8Array(w * h * 3) + for (let j = 0; j < h; j++) for (let i = 0; i < w; i++) { const q = ((p.y0 + j) * img.w + p.x0 + i) * 3; rgb.set(img.rgb.subarray(q, q + 3), (j * w + i) * 3) } + } + const r = await infer(c, ep, 1, rgb, w, h, log) + log(`готово за ${r.ms.toFixed(0)} мс`) + if (kind === 'sil') matte = r.plane; else depthRec = { plane: r.plane, w, h, crop: crop! } + work = null; schedule() + } catch (e) { log('ОШИБКА: ' + ((e as Error)?.message ?? e)) } finally { btn.disabled = false } +} + +// ---------- события ---------- +const PRESETS: Record> = { + // значения в мм детали; лицо — кроп морды ~40 мм, целиком — всё животное, мелочь теряется, поэтому деталь шире и сглаживание меньше + face: { silsrc: 'frame', dmode: 'engrave', dth: 0.15, wc: 30, wd: 40, wt: 30, partw: 40, dsm: 0.6, tsm: 0.3, fsm: 0.3, lcr: 1.4, lca: 1.1, slope: 1.5, bevel: 3, range: 2.4 }, + whole: { silsrc: 'model', dmode: 'engrave', dth: 0.18, wc: 25, wd: 45, wt: 30, partw: 70, dsm: 0.35, tsm: 0.15, fsm: 0.2, lcr: 0.9, lca: 0.9, slope: 2.5, bevel: 3, range: 2.8 }, + // рисунок/логотип: только силуэт по яркости, контурный скос, без глубины и деталей; кончики шипов сохраняются малым сглаживанием + drawing: { silsrc: 'brightness', dmode: 'engrave', dth: 0.2, wc: 55, wd: 0, wt: 45, partw: 60, bevel: 1.2, roll: 40, range: 1.8, fsm: 0.1, slope: 3, osm: 0.15, openmm2: 150, lca: 1.6, lcr: 0.7, tpre: 0.05, tsm: 0.05 }, +} +const applyPreset = (k: string) => { for (const [id, v] of Object.entries(PRESETS[k])) $(id).value = String(v); schedule() } +for (const k of Object.keys(PRESETS)) $('pre-' + k).addEventListener('click', () => applyPreset(k)) +$('runsil').addEventListener('click', () => runModel('sil')) +$('rundepth').addEventListener('click', () => runModel('depth')) +$('sample').addEventListener('change', () => { $('file').value = ''; const v = $('sample').value; if (chk('autopre')) applyPreset(v === 'ring' || v === 'check' ? 'drawing' : v === 'cat_face' ? 'face' : 'whole'); loadImage() }) +$('file').addEventListener('change', () => loadImage()) +$('face').addEventListener('click', () => { for (const [id, v] of [['cx', 42], ['cy', 34], ['cw', 42], ['ch', 24]] as const) $(id).value = String(v); schedule() }) +$('resetcrop').addEventListener('click', () => { for (const [id, v] of [['cx', 0], ['cy', 0], ['cw', 100], ['ch', 100]] as const) $(id).value = String(v); schedule() }) +for (const el of document.querySelectorAll('input[type=range], input[type=number], input[type=checkbox], input[type=radio], select')) { + if (['file', 'sample'].includes(el.id)) continue + el.addEventListener('input', () => { if (el.id === 'lightang') setLight(); schedule() }) +} +loadImage() diff --git a/poc/ai/models.ts b/poc/ai/models.ts new file mode 100644 index 0000000..5bfe136 --- /dev/null +++ b/poc/ai/models.ts @@ -0,0 +1,61 @@ +/** Кандидаты PoC. Ревизии закреплены (commit hash); размеры из API Hugging Face. */ +export interface Candidate { + id: string + label: string + url: string + bytes: number + license: string + size: number + mean: [number, number, number] + std: [number, number, number] + /** Нужно ли применять sigmoid к выходу (логиты). */ + sigmoid: boolean + /** Имя входа/выхода (null — первый). */ + input?: string + /** Только для сравнения качества: лицензия не позволяет включить в проект. */ + referenceOnly?: boolean + /** matte — маска объекта (по умолчанию), depth — относительная глубина (рельеф лица, а не силуэт). */ + kind?: 'matte' | 'depth' + /** Бэкенды, на которых модель реально работает (по замерам PoC); первый — предпочтительный. */ + backends?: ('wasm' | 'webgpu')[] + /** Внешние веса (.onnx_data) рядом с моделью. */ + data?: { path: string; url: string; bytes: number } +} +const hf = (repo: string, rev: string, file: string) => `https://huggingface.co/${repo}/resolve/${rev}/${file}` +const IMNET = { mean: [0.485, 0.456, 0.406] as [number, number, number], std: [0.229, 0.224, 0.225] as [number, number, number] } +const ZERO = { mean: [0, 0, 0] as [number, number, number], std: [1, 1, 1] as [number, number, number] } +const HALF = { mean: [0.5, 0.5, 0.5] as [number, number, number], std: [1, 1, 1] as [number, number, number] } + +/** Где модель работает по замерам владельца: остальные бэкенды дают ошибку, пустую маску или минуты. depth — не проверено, оба. */ +export const BACKENDS: Record = { + 'birefnet-lite-512-fp16': ['wasm'], 'birefnet-lite-512-fp32': ['wasm'], + 'ormbg-int8': ['wasm'], 'ormbg-fp16': ['wasm'], + 'inspyrenet-fp16': ['webgpu'], + 'isnet-imgly-fp16': ['webgpu'], 'isnet-imgly-fp32': ['webgpu'], + 'isnet-general-q': ['wasm', 'webgpu'], 'isnet-pim-q': ['wasm'], 'isnet-bw-fp32': ['wasm'], + 'u2netp': ['wasm'], 'u2net-bw-fp32': ['wasm'], + 'rmbg14-q': ['wasm', 'webgpu'], + 'depth-v2-small-q4f16': ['wasm', 'webgpu'], 'depth-v2-small-fp16': ['webgpu', 'wasm'], +} +export const backendsOf = (id: string): ('wasm' | 'webgpu')[] => BACKENDS[id] ?? ['wasm', 'webgpu'] + +export const CANDIDATES: Candidate[] = [ + { id: 'isnet-general-q', label: 'ISNet general int8 (Apache-2.0, 46 МБ)', url: hf('Ko033/isnet-general-use-onnx', '5349b617911fd60c619b52f32e2b593517b78df3', 'onnx/model_quantized.onnx'), bytes: 45902969, license: 'Apache-2.0', size: 1024, ...HALF, sigmoid: false }, + { id: 'u2netp', label: 'U²-Net-P (Apache-2.0, 4.6 МБ)', url: hf('Heliosoph/u2net-onnx', '7fc34deee10329bc039c10a73b98090d0c6f5c59', 'u2netp.onnx'), bytes: 4574861, license: 'Apache-2.0', size: 320, ...IMNET, sigmoid: false }, + { id: 'rmbg14-q', label: 'RMBG-1.4 int8 (Bria, NON-COMMERCIAL, 44 МБ) — эталон качества', url: hf('briaai/RMBG-1.4', '2ceba5a5efaec153162aedea169f76caf9b46cf8', 'onnx/model_quantized.onnx'), bytes: 44403226, license: 'bria-rmbg-1.4 (non-commercial)', size: 1024, ...HALF, sigmoid: false, referenceOnly: true }, + // второй раунд поиска (permissive-лицензии, ревизии закреплены) + { id: 'birefnet-lite-512-fp16', label: 'BiRefNet-lite 512 fp16 (MIT, 98 МБ)', url: hf('studioludens/birefnet-lite-512', '962e67969ec6acec9c19fab76acee4f5e22e990c', 'onnx/model_fp16.onnx'), bytes: 98484532, license: 'MIT', size: 512, ...IMNET, sigmoid: true }, + { id: 'birefnet-lite-512-fp32', label: 'BiRefNet-lite 512 fp32 (MIT, 192 МБ)', url: hf('studioludens/birefnet-lite-512', '962e67969ec6acec9c19fab76acee4f5e22e990c', 'onnx/model.onnx'), bytes: 191877254, license: 'MIT', size: 512, ...IMNET, sigmoid: true }, + { id: 'ormbg-int8', label: 'ormbg int8 (Apache-2.0, 44 МБ)', url: hf('onnx-community/ormbg-ONNX', '034e2d884afbab897e10e78fc5bb566b29533fd6', 'onnx/model_int8.onnx'), bytes: 44315136, license: 'Apache-2.0', size: 1024, ...ZERO, sigmoid: false }, + { id: 'ormbg-fp16', label: 'ormbg fp16 (Apache-2.0, 88 МБ)', url: hf('onnx-community/ormbg-ONNX', '034e2d884afbab897e10e78fc5bb566b29533fd6', 'onnx/model_fp16.onnx'), bytes: 88117930, license: 'Apache-2.0', size: 1024, ...ZERO, sigmoid: false }, + // третий раунд: убраны BiRefNet-lite 1024 (std::bad_alloc на обоих бэкендах у владельца) + { id: 'isnet-imgly-fp16', label: 'ISNet general imgly fp16 (MIT, 88 МБ)', url: hf('imgly/isnet-general-onnx', '440dea96dd4a3b06bbbf5abec3e26569dd7ec49f', 'onnx/model_fp16.onnx'), bytes: 88152708, license: 'MIT', size: 1024, mean: [0.502, 0.502, 0.502], std: [1.004, 1.004, 1.004], sigmoid: false }, + { id: 'isnet-imgly-fp32', label: 'ISNet general imgly fp32 (MIT, 176 МБ)', url: hf('imgly/isnet-general-onnx', '440dea96dd4a3b06bbbf5abec3e26569dd7ec49f', 'onnx/model.onnx'), bytes: 176149806, license: 'MIT', size: 1024, mean: [0.502, 0.502, 0.502], std: [1.004, 1.004, 1.004], sigmoid: false }, + { id: 'isnet-pim-q', label: 'ISNet pim-rmbg int8 (Apache-2.0, 44 МБ)', url: hf('drodriguezgdu/pim-rmbg-isnet', 'b6422ea5d8a184f0340f7a8dbf5c3a64eee95239', 'onnx/model_quantized.onnx'), bytes: 44229662, license: 'Apache-2.0', size: 1024, mean: [0.502, 0.502, 0.502], std: [1.004, 1.004, 1.004], sigmoid: false }, + { id: 'isnet-bw-fp32', label: 'IS-Net BritishWerewolf fp32 (Apache-2.0, 179 МБ)', url: hf('BritishWerewolf/IS-Net', '9783722d9f964c0286a411e7e8e6fede947d5a53', 'onnx/model.onnx'), bytes: 178648008, license: 'Apache-2.0', size: 1024, mean: [0.485, 0.456, 0.406], std: [1, 1, 1], sigmoid: false }, + { id: 'u2net-bw-fp32', label: 'U²-Net полная (Apache-2.0, 176 МБ)', url: hf('BritishWerewolf/U-2-Net', 'c08042b462485d8597c89d3ce831d00be8582c5c', 'onnx/model.onnx'), bytes: 175997641, license: 'Apache-2.0', size: 320, ...IMNET, sigmoid: false }, + { id: 'inspyrenet-fp16', label: 'InSPyReNet SwinB Plus Ultra fp16 (MIT, 199 МБ)', url: hf('OS-Software/InSPyReNet-SwinB-Plus-Ultra-ONNX', 'fe628d2d1465f775c707368a80248a2d5b5ad71f', 'onnx/model_fp16.onnx'), bytes: 199133027, license: 'MIT', size: 1024, ...IMNET, sigmoid: false }, + // карта глубины для рельефа (change 34): выход — относительная глубина, а не маска + { id: 'depth-v2-small-q4f16', kind: 'depth', label: 'Depth Anything V2 Small q4f16 (Apache-2.0, 20 МБ) — глубина', url: hf('onnx-community/depth-anything-v2-small-ONNX', 'c3b67641fd837b2368757101311e5d21e511441e', 'onnx/model_q4f16.onnx'), bytes: 212397, data: { path: 'model_q4f16.onnx_data', url: hf('onnx-community/depth-anything-v2-small-ONNX', 'c3b67641fd837b2368757101311e5d21e511441e', 'onnx/model_q4f16.onnx_data'), bytes: 20200448 }, license: 'Apache-2.0', size: 518, ...IMNET, sigmoid: false }, + { id: 'depth-v2-small-fp16', kind: 'depth', label: 'Depth Anything V2 Small fp16 (Apache-2.0, 50 МБ) — глубина', url: hf('onnx-community/depth-anything-v2-small-ONNX', 'c3b67641fd837b2368757101311e5d21e511441e', 'onnx/model_fp16.onnx'), bytes: 180471, data: { path: 'model_fp16.onnx_data', url: hf('onnx-community/depth-anything-v2-small-ONNX', 'c3b67641fd837b2368757101311e5d21e511441e', 'onnx/model_fp16.onnx_data'), bytes: 50392064 }, license: 'Apache-2.0', size: 518, ...IMNET, sigmoid: false }, +] diff --git a/poc/ai/outline.ts b/poc/ai/outline.ts new file mode 100644 index 0000000..899480c --- /dev/null +++ b/poc/ai/outline.ts @@ -0,0 +1,184 @@ +/** Векторный контур для PoC: трассировка, сглаживание вдоль кривой с сохранением острых углов, знаковое расстояние до полигонов. */ +import { traceAlpha } from '../../src/geometry/textOutline' + +export type Pt = [number, number] + +const len = (a: Pt, b: Pt) => Math.hypot(b[0] - a[0], b[1] - a[1]) + +/** Равномерная передискретизация замкнутой петли шагом h. */ +export function resample(loop: Pt[], h: number): Pt[] { + const n = loop.length, cum = [0] + for (let i = 0; i < n; i++) cum.push(cum[i] + len(loop[i], loop[(i + 1) % n])) + const L = cum[n], N = Math.max(8, Math.round(L / h)), out: Pt[] = [] + let k = 0 + for (let i = 0; i < N; i++) { + const d = (i / N) * L + while (cum[k + 1] < d) k++ + const t = (d - cum[k]) / Math.max(1e-9, cum[k + 1] - cum[k]), a = loop[k % n], b = loop[(k + 1) % n] + out.push([a[0] + (b[0] - a[0]) * t, a[1] + (b[1] - a[1]) * t]) + } + return out +} + +/** Углы: вершины, где направление кривой поворачивается больше cornerDeg на окне ±win (локальный максимум поворота). */ +export function findCorners(p: Pt[], win: number, cornerDeg: number): boolean[] { + const n = p.length, turn = new Float64Array(n) + for (let i = 0; i < n; i++) { + const a = p[(i - win + n) % n], b = p[i], c = p[(i + win) % n] + const v1x = b[0] - a[0], v1y = b[1] - a[1], v2x = c[0] - b[0], v2y = c[1] - b[1] + turn[i] = Math.abs(Math.atan2(v1x * v2y - v1y * v2x, v1x * v2x + v1y * v2y)) * 180 / Math.PI + } + const corner = new Array(n).fill(false) + for (let i = 0; i < n; i++) { + if (turn[i] < cornerDeg) continue + let max = true + for (let d = -win; d <= win; d++) if (turn[(i + d + n) % n] > turn[i] || (turn[(i + d + n) % n] === turn[i] && d < 0)) { max = false; break } + corner[i] = max + } + return corner +} + +const gaussKernel = (sigma: number) => { const r = Math.max(1, Math.ceil(3 * sigma)), k: number[] = []; let s = 0; for (let i = -r; i <= r; i++) { const v = Math.exp(-(i * i) / (2 * sigma * sigma)); k.push(v); s += v } return { r, k: k.map((v) => v / s) } } + +/** Сглаживание гауссом вдоль петли; между углами концы закреплены (нечётное отражение), без углов — по кольцу. */ +export function smoothLoop(p: Pt[], sigma: number, corner: boolean[]): Pt[] { + if (sigma < 0.3) return p + const n = p.length, { r, k } = gaussKernel(sigma), ids = corner.map((c, i) => (c ? i : -1)).filter((i) => i >= 0) + const out: Pt[] = p.map((q) => [q[0], q[1]]) + const smoothRun = (idx: number[], fixedEnds: boolean) => { // idx — индексы вершин участка по порядку + const m = idx.length + const at = (t: number, axis: 0 | 1): number => { + if (!fixedEnds) return p[idx[((t % m) + m) % m]][axis] + if (t < 0) return 2 * p[idx[0]][axis] - p[idx[Math.min(m - 1, -t)]][axis] + if (t > m - 1) return 2 * p[idx[m - 1]][axis] - p[idx[Math.max(0, 2 * (m - 1) - t)]][axis] + return p[idx[t]][axis] + } + for (let t = fixedEnds ? 1 : 0; t < (fixedEnds ? m - 1 : m); t++) { + let x = 0, y = 0 + for (let d = -r; d <= r; d++) { x += k[d + r] * at(t + d, 0); y += k[d + r] * at(t + d, 1) } + out[idx[t]] = [x, y] + } + } + if (!ids.length) { smoothRun(Array.from({ length: n }, (_, i) => i), false); return out } + for (let c = 0; c < ids.length; c++) { + const a = ids[c], b = ids[(c + 1) % ids.length], idx: number[] = [] + for (let i = a; ; i = (i + 1) % n) { idx.push(i); if (i === b && idx.length > 1) break; if (idx.length > n + 1) break } + if (idx.length > 2) smoothRun(idx, true) + } + return out +} + +/** Контуры мягкого поля (значения 0…255, уровень 128): внешние и внутренние петли, сглаженные с сохранением углов. Единицы — пиксели растра. */ +export function vectorOutline(field255: Uint8Array, w: number, h: number, sigmaPx: number, cornerDeg: number, stepPx = 0.7): Pt[][] { + const loops = traceAlpha(field255, w, h, 128) + return loops.map((l) => { + if (l.length < 4) return l as Pt[] + const rs = resample(l as Pt[], stepPx), win = Math.max(2, Math.round(1.5 / stepPx)) + return smoothLoop(rs, sigmaPx / stepPx, findCorners(rs, win, cornerDeg)) + }) +} + +/** Расстояние от точки до набора петель и чётность пересечений (внутри ли). */ +export function signedDistance(loops: Pt[][], x: number, y: number): number { + let best = Infinity, cross = 0 + for (const l of loops) { + const n = l.length + for (let i = 0; i < n; i++) { + const a = l[i], b = l[(i + 1) % n], dx = b[0] - a[0], dy = b[1] - a[1], t = Math.max(0, Math.min(1, ((x - a[0]) * dx + (y - a[1]) * dy) / (dx * dx + dy * dy || 1))) + const d = Math.hypot(x - (a[0] + dx * t), y - (a[1] + dy * t)); if (d < best) best = d + if ((a[1] > y) !== (b[1] > y) && x < a[0] + ((y - a[1]) / (b[1] - a[1])) * dx) cross++ + } + } + return cross & 1 ? best : -best +} + +// ---------- подгонка кубических кривых Безье (Шнейдер, «Graphics Gems»): минимум точек, гладкие стенки ---------- +export type V2 = [number, number] +export type Bez = [V2, V2, V2, V2] +const vadd = (a: V2, b: V2): V2 => [a[0] + b[0], a[1] + b[1]] +const vsub = (a: V2, b: V2): V2 => [a[0] - b[0], a[1] - b[1]] +const vmul = (a: V2, s: number): V2 => [a[0] * s, a[1] * s] +const vdot = (a: V2, b: V2) => a[0] * b[0] + a[1] * b[1] +const vlen = (a: V2) => Math.hypot(a[0], a[1]) +const vnorm = (a: V2): V2 => { const l = vlen(a) || 1; return [a[0] / l, a[1] / l] } +const B0 = (u: number) => (1 - u) ** 3, B1 = (u: number) => 3 * u * (1 - u) ** 2, B2 = (u: number) => 3 * u * u * (1 - u), B3 = (u: number) => u ** 3 +export const bezAt = (c: Bez, t: number): V2 => [B0(t) * c[0][0] + B1(t) * c[1][0] + B2(t) * c[2][0] + B3(t) * c[3][0], B0(t) * c[0][1] + B1(t) * c[1][1] + B2(t) * c[2][1] + B3(t) * c[3][1]] +export const bezTan = (c: Bez, t: number): V2 => { + const a = vmul(vsub(c[1], c[0]), 3 * (1 - t) ** 2), b = vmul(vsub(c[2], c[1]), 6 * (1 - t) * t), d = vmul(vsub(c[3], c[2]), 3 * t * t) + const r = vadd(vadd(a, b), d); return vlen(r) < 1e-9 ? vnorm(vsub(c[3], c[0])) : vnorm(r) +} +const bezD2 = (c: Bez, t: number): V2 => { + const a = vmul(vadd(vsub(c[2], vmul(c[1], 2)), c[0]), 6 * (1 - t)), b = vmul(vadd(vsub(c[3], vmul(c[2], 2)), c[1]), 6 * t); return vadd(a, b) +} +const bezD1 = (c: Bez, t: number): V2 => vadd(vadd(vmul(vsub(c[1], c[0]), 3 * (1 - t) ** 2), vmul(vsub(c[2], c[1]), 6 * (1 - t) * t)), vmul(vsub(c[3], c[2]), 3 * t * t)) + +function chordParam(d: V2[], a: number, b: number): number[] { + const u = [0]; for (let i = a + 1; i <= b; i++) u.push(u[u.length - 1] + vlen(vsub(d[i], d[i - 1]))) + const L = u[u.length - 1] || 1; return u.map((x) => x / L) +} +function genBezier(d: V2[], a: number, b: number, u: number[], t1: V2, t2: V2): Bez { + let c00 = 0, c01 = 0, c11 = 0, x0 = 0, x1 = 0 + for (let i = 0; i < u.length; i++) { + const A0 = vmul(t1, B1(u[i])), A1 = vmul(t2, B2(u[i])) + c00 += vdot(A0, A0); c01 += vdot(A0, A1); c11 += vdot(A1, A1) + const tmp = vsub(d[a + i], vadd(vadd(vmul(d[a], B0(u[i]) + B1(u[i])), vmul(d[b], B2(u[i]) + B3(u[i]))), [0, 0])) + x0 += vdot(A0, tmp); x1 += vdot(A1, tmp) + } + const det = c00 * c11 - c01 * c01, al = det === 0 ? 0 : (x0 * c11 - x1 * c01) / det, ar = det === 0 ? 0 : (c00 * x1 - c01 * x0) / det + const seg = vlen(vsub(d[b], d[a])), eps = 1e-6 * seg + if (al < eps || ar < eps) { const k = seg / 3; return [d[a], vadd(d[a], vmul(t1, k)), vadd(d[b], vmul(t2, k)), d[b]] } + return [d[a], vadd(d[a], vmul(t1, al)), vadd(d[b], vmul(t2, ar)), d[b]] +} +function maxErr(d: V2[], a: number, b: number, c: Bez, u: number[]): { e: number; split: number } { + let e = 0, split = Math.floor((a + b) / 2) + for (let i = 1; i < u.length - 1; i++) { const p = bezAt(c, u[i]), dd = vsub(p, d[a + i]), s = vdot(dd, dd); if (s >= e) { e = s; split = a + i } } + return { e, split } +} +function reparam(d: V2[], a: number, c: Bez, u: number[]): number[] { + return u.map((ui, i) => { + const p = bezAt(c, ui), d1 = bezD1(c, ui), d2 = bezD2(c, ui), diff = vsub(p, d[a + i]), num = vdot(diff, d1), den = vdot(d1, d1) + vdot(diff, d2) + return den === 0 ? ui : Math.min(1, Math.max(0, ui - num / den)) + }) +} +function fitCubic(d: V2[], a: number, b: number, t1: V2, t2: V2, err: number, out: Bez[]) { + if (b - a === 1) { const k = vlen(vsub(d[b], d[a])) / 3; out.push([d[a], vadd(d[a], vmul(t1, k)), vadd(d[b], vmul(t2, k)), d[b]]); return } + let u = chordParam(d, a, b), c = genBezier(d, a, b, u, t1, t2), { e, split } = maxErr(d, a, b, c, u) + if (e < err * err) { out.push(c); return } + if (e < err * err * 16) for (let k = 0; k < 6; k++) { + u = reparam(d, a, c, u); c = genBezier(d, a, b, u, t1, t2); const r = maxErr(d, a, b, c, u); e = r.e; split = r.split + if (e < err * err) { out.push(c); return } + } + const tc = vnorm(vsub(d[split - 1], d[split + 1])) + fitCubic(d, a, split, t1, tc, err, out); fitCubic(d, split, b, vmul(tc, -1), t2, err, out) +} + +/** Замкнутый контур (точки в мм) → цепочка кубических Безье с допуском tol; в острых углах (по `corner`) разрыв касательной, между ними G1. */ +export function fitLoop(pts: V2[], corner: boolean[], tol: number): { segs: Bez[]; cornerAt: boolean[] } { + const n = pts.length, ids = corner.map((c, i) => (c ? i : -1)).filter((i) => i >= 0) + const w = 3, tan = (i: number): V2 => vnorm(vsub(pts[(i + w) % n], pts[(i - w + n) % n])) + const cuts = ids.length ? ids : [0, Math.floor(n / 2)] + const segs: Bez[] = [], cornerAt: boolean[] = [] + for (let c = 0; c < cuts.length; c++) { + const a = cuts[c], b = cuts[(c + 1) % cuts.length], run: V2[] = [] + for (let i = a; ; i = (i + 1) % n) { run.push(pts[i]); if (i === b && run.length > 1) break; if (run.length > n + 1) break } + if (run.length < 2) continue + const m = run.length, k = Math.min(w, m - 1) + const isCorner = ids.length > 0 + const t1 = isCorner ? vnorm(vsub(run[k], run[0])) : tan(a), t2 = isCorner ? vnorm(vsub(run[m - 1 - k], run[m - 1])) : vmul(tan(b), -1) + const before = segs.length + fitCubic(run, 0, m - 1, t1, t2, tol, segs) + for (let s = before; s < segs.length; s++) cornerAt.push(s === before && isCorner) + } + return { segs, cornerAt } +} + +/** Выборка кривых с шагом ≈ step мм: точки и единичные нормали наружу (контур против часовой). cornerAt — начала сегментов с разрывом нормали. */ +export function tessellate(segs: Bez[], step: number): { p: V2; n: V2; start: boolean; seg: number }[] { + const out: { p: V2; n: V2; start: boolean; seg: number }[] = [] + segs.forEach((c, si) => { + let L = 0, prev = c[0]; for (let i = 1; i <= 16; i++) { const q = bezAt(c, i / 16); L += vlen(vsub(q, prev)); prev = q } + const n = Math.max(2, Math.ceil(L / step)) + for (let i = 0; i < n; i++) { const t = i / n, tg = bezTan(c, t); out.push({ p: bezAt(c, t), n: [tg[1], -tg[0]], start: i === 0, seg: si }) } + }) + return out +} diff --git a/poc/ai/samples/cat.jpeg b/poc/ai/samples/cat.jpeg new file mode 100644 index 0000000..da385ff Binary files /dev/null and b/poc/ai/samples/cat.jpeg differ diff --git a/poc/ai/samples/cat_face.jpeg b/poc/ai/samples/cat_face.jpeg new file mode 100644 index 0000000..c7507fe Binary files /dev/null and b/poc/ai/samples/cat_face.jpeg differ diff --git a/poc/ai/samples/check.jpeg b/poc/ai/samples/check.jpeg new file mode 100644 index 0000000..a9d84a9 Binary files /dev/null and b/poc/ai/samples/check.jpeg differ diff --git a/poc/ai/samples/flash.jpeg b/poc/ai/samples/flash.jpeg new file mode 100644 index 0000000..26b7bbd Binary files /dev/null and b/poc/ai/samples/flash.jpeg differ diff --git a/poc/ai/samples/noflash.jpeg b/poc/ai/samples/noflash.jpeg new file mode 100644 index 0000000..a19645b Binary files /dev/null and b/poc/ai/samples/noflash.jpeg differ diff --git a/poc/ai/samples/ring.jpeg b/poc/ai/samples/ring.jpeg new file mode 100644 index 0000000..24fd8e2 Binary files /dev/null and b/poc/ai/samples/ring.jpeg differ diff --git a/poc/ai/vite.config.ts b/poc/ai/vite.config.ts new file mode 100644 index 0000000..a5d5867 --- /dev/null +++ b/poc/ai/vite.config.ts @@ -0,0 +1,14 @@ +import { defineConfig } from 'vite' + +// PoC нейросетевых силуэтов (change 32). Не часть приложения: отдельный dev-сервер, `npm run poc:ai`. +// POC_COI=1 включает кросс-изоляцию (COOP/COEP), нужную для многопоточного WASM: так проверяется, стоит ли городить обходку для GitHub Pages. +const coi = process.env.POC_COI === '1' +export default defineConfig({ + root: 'poc/ai', + base: './', + server: { + port: 5198, strictPort: true, fs: { allow: ['../..'] }, + headers: coi ? { 'Cross-Origin-Opener-Policy': 'same-origin', 'Cross-Origin-Embedder-Policy': 'credentialless' } : {}, + }, + optimizeDeps: { exclude: ['onnxruntime-web'] }, +})