Skip to content

Make personal learning incremental - #28

Merged
enaboapps merged 3 commits into
mainfrom
claude/incremental-learning
Oct 12, 2026
Merged

enaboapps merged 3 commits into
mainfrom
claude/incremental-learning

Conversation

@enaboapps

@enaboapps enaboapps commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

Closes #27

Predictor::learn and import used to clone the whole baseline and reload every personal count on each call. Now, when PRAGMA data_version shows that no other connection has committed since this predictor's snapshot, they add only the new counts to the in-memory snapshot, with the same 5× weight. Otherwise they reload and validate everything, as before. The fast path also requires the personal database to have exactly the schema initialize creates. Triggers, replacing constraints, foreign-key actions and other changes can give the connection's own write side effects that only the full reload validates. A failed commit clears the recorded version, so the next write reloads. A connection's own commits leave its data_version unchanged. The version is read inside the transaction, so no fallible work follows the commit. refresh_personal and reset_personal record the version too. The schema, API and prediction maths are unchanged.

Results

Production en-aac-oanc-v1 baseline, release build, 50 sentences (ignored test learning_cost_on_production_baseline):

Median Max
Before 191 ms 624 ms
After 8.4 ms 11 ms

Validation

  • incremental_learning_matches_a_full_reload: after each learned text, the ranked words and score bits match a freshly opened predictor.
  • learning_after_another_writer_includes_its_counts: a second writer's counts, and a reset by another instance, are picked up through the full-reload path. Results match a reload throughout.
  • A unit test shows the fast path runs for this connection's own writes and its own reset. It also shows that another writer, or a non-standard schema, forces a reload.
  • a_trigger_side_effect_is_still_validated_before_commit and a_replacing_constraint_is_still_validated_before_commit check that both schema-based cases are still refused and rolled back, as on main.
  • duplicate_imports_and_empty_text_match_a_full_reload.
  • Existing concurrency, import-dedup, reset and corruption tests pass unchanged.
  • Three independent reviews. The final one found no issues. It confirmed that personal databases created by v0.1.0–v0.2.1 match the standard schema, and it fuzzed word normalization (300k strings).
  • cargo fmt --check, cargo clippy --locked --all-targets -D warnings and cargo test --locked pass.

This is independent of #26. Switchify PC needs this before it can learn while someone types.

🤖 Generated with Claude Code

Each learn cloned the full baseline and reloaded every personal count, about
190 ms per sentence on the production baseline. When PRAGMA data_version shows
no other connection has committed since the snapshot, add only the new counts
with the existing weight. Otherwise reload and validate as before. Predictions
and scores stay bit-identical to a full reload. Median cost is now about 9 ms.

Closes #27

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@enaboapps enaboapps added this to the v0.2.2 milestone Oct 12, 2026
OwenMcGirr and others added 2 commits October 12, 2026 12:31
A trigger can give the connection's own write side effects that only a full
reload validates, so learning takes the reload path whenever one exists, as
before. A failed commit clears the recorded data_version, so the next write
reloads. Unit tests show the fast path runs, and when it does not.

Refs #27

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Constraints such as UNIQUE ... ON CONFLICT REPLACE, foreign-key actions or a
missing CHECK can also give the connection's own write effects beyond its
counts. Learning adds counts in place only when the personal database has
exactly the schema initialize creates; anything else reloads and validates.

Refs #27

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
enaboapps pushed a commit that referenced this pull request Oct 12, 2026
The in-place update test now covers words the baseline trusts, an import
and a duplicate import. The timing test also times a learn that makes a new
word usable. Measured against #28 on the same workload, prediction and
learning costs match within noise, so the docs say so.

Refs #29

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Oct 12, 2026
@enaboapps
enaboapps marked this pull request as ready for review October 12, 2026 15:53
@enaboapps

Copy link
Copy Markdown
Contributor Author

Three independent reviews; the last found no issues. All 15 CI checks pass. #30 is stacked on this branch, so don't delete claude/incremental-learning on merge, or #30 will close.

@enaboapps
enaboapps merged commit 4ff155f into main Oct 12, 2026
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Make personal learning incremental

2 participants