From 95710db7c1ee85f310ac10af0c3bc7a0c3ee4500 Mon Sep 17 00:00:00 2001 From: Owen McGirr Date: Mon, 12 Oct 2026 14:22:06 +0100 Subject: [PATCH 1/8] Weight learned words 500 times so they are suggested At 5x, learned counts were negligible beside the baseline: on a synthetic person, learned words reached the six slots for 16.7% of queries. At 500x they reach 53.8%, while general English with that learning present is unchanged (71.0% without learning, 71.5% and 71.3% light and heavy). Adds the synthetic fixture and evaluator to reproduce this. Closes #29 Co-Authored-By: Claude Opus 5.5 --- README.md | 6 +- docs/personal-learning.md | 56 ++++++++++++++++++ neural/fixtures/personal.json | 72 +++++++++++++++++++++++ scripts/personal_evaluate.py | 106 ++++++++++++++++++++++++++++++++++ src/lib.rs | 10 +++- tests/prediction.rs | 3 +- 6 files changed, 247 insertions(+), 6 deletions(-) create mode 100644 docs/personal-learning.md create mode 100644 neural/fixtures/personal.json create mode 100644 scripts/personal_evaluate.py diff --git a/README.md b/README.md index 14dce0e..294442c 100644 --- a/README.md +++ b/README.md @@ -128,8 +128,10 @@ fn example() -> Result<()> { Fallible operations return typed `Error` values. Reuse a long-lived predictor; keep loading, imports, learning, refresh and resets on a worker thread. Prediction uses memory only. The model uses unigram/bigram/trigram weights **0.1/0.3/0.6**, -renormalized over available contexts, and combines baseline counts with **5×** -personal counts before probability calculation. Prefix matches rank by score, +renormalized over available contexts, and combines baseline counts with **500×** +personal counts before probability calculation. Learned text is tiny beside the +baseline, so a lower weight leaves personal words almost never suggested; see +[personal learning](docs/personal-learning.md). Prefix matches rank by score, then alphabetically. Schema v1 and existing personal databases remain compatible. ## Validation and model selection diff --git a/docs/personal-learning.md b/docs/personal-learning.md new file mode 100644 index 0000000..06ef302 --- /dev/null +++ b/docs/personal-learning.md @@ -0,0 +1,56 @@ +# Personal learning weight + +Issue #29 raises the weight of learned counts from 5 to 500 baseline +occurrences each. Learned text is tiny beside the baseline. At 5×, a name or care +term typed a few times hardly ever reaches the suggestions. + +## Measurement + +`scripts/personal_evaluate.py` uses `neural/fixtures/personal.json`. The fixture +is synthetic text imitating one person's typing; it is not user data. It has 40 +learned sentences and 25 held-out test sentences that reuse their names, places, +care words and interests. Each test word is queried with 0–2 typed letters, 445 +queries in all. 78 of them are personal words: learned and rare in the baseline. +Both six slots count. + +| Personal weight | Six-slot top six | Personal words | +| ---: | ---: | ---: | +| no learning | 51.0% | 1.3% | +| 5 | 54.6% | 16.7% | +| 50 | 57.1% | 28.2% | +| **500** | **61.8%** | **53.8%** | +| 2,000 | 62.9% | 60.3% | +| 5,000 | 64.0% | 65.4% | + +General English must not suffer for someone who has learned text. 600 queries +from the frozen general workload, with natural context, were run with that +person's learning present. "Light" is the 40 sentences learned once; "heavy" is +the same text learned ten times. + +| Personal weight | Light | Heavy | +| ---: | ---: | ---: | +| no learning | 71.0% | | +| 5 | 71.2% | 71.2% | +| **500** | **71.5%** | **71.3%** | +| 2,000 | 71.5% | 71.0% | +| 5,000 | 71.3% | 71.0% | + +Gains continue above 500 but slow, and the test sentences reuse exactly the +learned vocabulary, which favours high weights. 500 keeps most of the gain with +no measurable cost to general text. Differences of under about two points on 600 +queries are within noise. + +Offering learned words to the neural generator as extra candidates was also +tried. It added under half a point at 500 and nearly doubled generation +latency, so it is not included. + +Measured on Windows 11 with an AMD Ryzen AI 9 HX 370. These are synthetic +regression measurements, not evidence of real users' accuracy. + +## Reproduce + +Build the root CLI for learning and the neural workspace, then run: + +```text +python scripts/personal_evaluate.py --cli --learner --baseline --bundle --worker [--accelerated-worker ] --output +``` diff --git a/neural/fixtures/personal.json b/neural/fixtures/personal.json new file mode 100644 index 0000000..4b90a2c --- /dev/null +++ b/neural/fixtures/personal.json @@ -0,0 +1,72 @@ +{ + "note": "Synthetic text written for this evaluation, imitating one person's typing with their own names, places, care words and interests. It is not real user data. `learn` is learned into a personal database; `test` is held out and reuses the same vocabulary in new sentences.", + "learn": [ + "Siobhan is coming at ten to help with the hoist.", + "Can you ask Siobhan to bring my blue cardigan?", + "Aoife phoned last night about the physio appointment.", + "My physio session with Declan is on Thursday.", + "Declan says my shoulder is getting stronger.", + "Please check the suction machine is charged.", + "The suction tubing needs changing today.", + "Can someone set up my PEG feed at six?", + "The PEG site looks a bit red this morning.", + "I want to watch Liverpool tonight.", + "Salah scored again for Liverpool!", + "Did you see the Liverpool match on Saturday?", + "Cathal is driving us to Portrush on Sunday.", + "I love the beach at Portrush when it is sunny.", + "We stopped at Ballymena for lunch.", + "Mum lives in Ballymena near the park.", + "Niamh sent photos of Biscuit chasing the ball.", + "Biscuit needs his walk before tea.", + "Can you feed Biscuit please?", + "I am working on my crochet blanket.", + "The crochet hook fell under the chair.", + "Coronation Street is on at half seven.", + "Turn Coronation Street up a bit please.", + "Dr Mulholland wants to see me next week.", + "Ask Dr Mulholland about my new tablets.", + "My wheelchair battery is low again.", + "The wheelchair footplate is loose.", + "Can you tilt my wheelchair back a little?", + "Siobhan and Aoife are both on shift tomorrow.", + "I need my glasses and the iPad please.", + "Please charge the iPad before bed.", + "Cathal made a lovely stew for dinner.", + "Tell Niamh I said happy birthday.", + "I would like a cup of tea with two sugars.", + "My carer is running late this morning.", + "The carer from the agency was really kind.", + "Can you open the curtains please?", + "I am too warm, can you turn the heating down?", + "Thank you, that is much better.", + "I love you all very much." + ], + "test": [ + "Is Siobhan working this weekend?", + "Please ring Aoife and say I am fine.", + "Declan wants me to do my stretches.", + "Can you clean the suction machine?", + "Please flush my PEG after the feed.", + "Liverpool are playing at home tomorrow.", + "Put the Liverpool game on please.", + "Can Cathal take me to Portrush next week?", + "We should visit Mum in Ballymena soon.", + "Has anyone walked Biscuit today?", + "Niamh is bringing Biscuit over later.", + "Pass me my crochet bag please.", + "Record Coronation Street for me.", + "When is my appointment with Dr Mulholland?", + "My wheelchair cushion needs pumping up.", + "The hoist sling is in the cupboard.", + "My physio exercises are hard today.", + "Can you charge my wheelchair tonight?", + "Ask Siobhan if she can stay for tea.", + "I would like to talk to Cathal.", + "The iPad volume is too low.", + "My carer helped me with the hoist.", + "Can you get the suction ready please?", + "Aoife says the physio is cancelled.", + "I am tired, can I go to bed?" + ] +} diff --git a/scripts/personal_evaluate.py b/scripts/personal_evaluate.py new file mode 100644 index 0000000..a431987 --- /dev/null +++ b/scripts/personal_evaluate.py @@ -0,0 +1,106 @@ +#!/usr/bin/env python3 +"""Measure what learning adds to six-slot prediction, on a synthetic person. + +A temporary personal database learns neural/fixtures/personal.json `learn` text +through the CLI. Held-out `test` sentences are then queried with and without +learning. The fixture is synthetic, written for this evaluation; results show +the mechanism, not real users' accuracy. +Reports contain counts and timing, never words. Requires psutil and regex. +""" +import argparse +import json +from pathlib import Path +import subprocess +import tempfile +import time + +from neural_evaluate import Client, ROOT, timing, words + +FIXTURE = ROOT / 'neural/fixtures/personal.json' + + +def queries(baseline_counts): + import regex + fixture = json.loads(FIXTURE.read_bytes()) + learned = {w for sentence in fixture['learn'] for w in words(sentence)} + result = [] + for sentence in fixture['test']: + text = sentence.replace('’', "'") + for match in regex.finditer(r"\p{L}[\p{L}\p{M}]*(?:'\p{L}[\p{L}\p{M}]*)*", text): + target = words(match.group())[0] + graphemes = regex.findall(r'\X', target) + for n in range(min(3, len(graphemes))): + # Personal: learned, and rare or unknown in the baseline. + own = target in learned and baseline_counts.get(target, 0) < 10 + result.append((text[:match.start()], ''.join(graphemes[:n]), target, own)) + return result + + +def run(args, personal, items): + command = [str(args.cli), 'stream', '--baseline', str(args.baseline), '--bundle', str(args.bundle), + '--worker', str(args.worker)] + if args.accelerated_worker: + command += ['--accelerated-worker', str(args.accelerated_worker)] + if personal: + command += ['--personal', str(personal)] + client = Client(command) + hits = dict(all=0, personal=0, generated=0) + times = [] + try: + for before, prefix, target, own in items: + client.process.stdin.write('{"command":"reset"}\n') + client.process.stdin.flush() + assert client.receive()['type'] == 'reset' + start = time.perf_counter() + client.process.stdin.write(json.dumps(dict(command='generate', before=before, + prefix=prefix, session=1)) + '\n') + client.process.stdin.flush() + immediate = client.receive() + instant = immediate['result']['words'] + generated = [] + if immediate['result']['refinement_requested']: + reply = client.receive() + if reply['type'] == 'refined': + generated = reply['result']['words'] + times.append((time.perf_counter() - start) * 1000) + hit = target in instant + generated + hits['all'] += hit + hits['personal'] += hit and own + hits['generated'] += target in generated + finally: + client.close() + count = len(items) + own = sum(item[3] for item in items) + return dict(top6=hits['all'] / count, personal_top6=hits['personal'] / own, + generated_hits=hits['generated'] / count, latency=timing(times)) + + +def main(): + parser = argparse.ArgumentParser(description=__doc__) + for name in ['cli', 'learner', 'baseline', 'bundle', 'worker', 'output']: + parser.add_argument('--' + name, type=Path, required=True) + parser.add_argument('--accelerated-worker', type=Path) + args = parser.parse_args() + import sqlite3 + db = sqlite3.connect(args.baseline.resolve().as_uri() + '?mode=ro', uri=True) + baseline_counts = dict(db.execute("SELECT word, count FROM counts WHERE context='[]'")) + db.close() + items = queries(baseline_counts) + with tempfile.TemporaryDirectory() as directory: + personal = Path(directory) / 'personal.sqlite' + # Newlines separate context exactly as separate learn calls would. + text = '\n'.join(json.loads(FIXTURE.read_bytes())['learn']) + subprocess.run([str(args.learner), 'learn', '--baseline', str(args.baseline), + '--personal', str(personal)], input=text, text=True, + check=True, capture_output=True) + report = dict( + queries=len(items), personal_queries=sum(item[3] for item in items), + without_learning=run(args, None, items), + with_learning=run(args, personal, items), + note='Synthetic fixture; shows the mechanism, not real-user accuracy.') + args.output.write_text(json.dumps(report, indent=2) + '\n', encoding='utf-8') + print(json.dumps(report, indent=2)) + + +if __name__ == '__main__': + main() diff --git a/src/lib.rs b/src/lib.rs index 1fdae8e..b46fbc3 100644 --- a/src/lib.rs +++ b/src/lib.rs @@ -345,6 +345,10 @@ impl Default for Options { } } } +/// How many baseline occurrences one learned occurrence counts as. Learned +/// text is tiny beside the baseline, so a lower weight leaves names and other +/// personal words almost never suggested; see docs/personal-learning.md. +const PERSONAL_WEIGHT: i64 = 500; pub struct Predictor { baseline: Counts, combined: Counts, @@ -434,7 +438,7 @@ impl Predictor { if check(&tx)? != "personal" { return Err(Error::Invalid("expected personal database".into())); } - combined.merge(&load_checked(&tx, "personal")?, 5); + combined.merge(&load_checked(&tx, "personal")?, PERSONAL_WEIGHT); let version = data_version(&tx)?; if let Err(error) = tx.commit() { // The outcome is unknown, so the next write reloads. @@ -524,7 +528,7 @@ impl Predictor { None } else { let mut combined = self.baseline.clone(); - combined.merge(&load_checked(&tx, "personal")?, 5); + combined.merge(&load_checked(&tx, "personal")?, PERSONAL_WEIGHT); Some(combined) }; if let Err(error) = tx.commit() { @@ -535,7 +539,7 @@ impl Predictor { self.seen = Some(version); match combined { Some(combined) => self.combined = combined, - None if imported => self.combined.merge(&delta, 5), + None if imported => self.combined.merge(&delta, PERSONAL_WEIGHT), None => {} } Ok(imported) diff --git a/tests/prediction.rs b/tests/prediction.rs index 8832528..0dbadb7 100644 --- a/tests/prediction.rs +++ b/tests/prediction.rs @@ -240,7 +240,8 @@ fn interpolation_renormalizes_and_personal_counts_are_weighted_before_probabilit assert!((backed_off - (0.3 * 0.5 + 0.1 / 9.0) / 0.4).abs() < 1e-12); p.learn("i like tea.").unwrap(); let learned = p.predict("i like ", "tea", opts).remove(0).score; - assert!((learned - (0.9 * 6.0 / 7.0 + 0.1 * 6.0 / 24.0)).abs() < 1e-12); + // One learned occurrence counts as 500 baseline ones. + assert!((learned - (0.9 * 501.0 / 502.0 + 0.1 * 501.0 / 1509.0)).abs() < 1e-12); let total: f64 = p.predict("i like ", "", opts).iter().map(|s| s.score).sum(); assert!((total - 1.0).abs() < 1e-12); } From 3e9234cda06103b2042cb08591a6889a11f4c710 Mon Sep 17 00:00:00 2001 From: Owen McGirr Date: Mon, 12 Oct 2026 15:22:14 +0100 Subject: [PATCH 2/8] Keep learned counts apart and weight them boundedly Learned counts were added to a copy of the baseline at a fixed weight. A fixed 500x let one misspelling become the top suggestion, let heavy use outgrow the baseline, and could overflow on crafted counts. Now: - learned counts are kept apart and combined when predicting, in floating point, so stored counts cannot overflow when weighted; - one learned occurrence counts as 500 / (1 + learned words / 2000), so learned counts approach a bounded share of the baseline; - a word the baseline lacks is suggested once learned twice; - the baseline is no longer copied: peak predictor memory falls from 362 MiB to 134 MiB, with or without a personal database. With no or an empty personal database, predictions are bit-identical. On the synthetic person with four misspellings, top six rises from 50.3% to 60.4%, personal words from 1.3% to 46.2%, and no misspelling is suggested. Refs #29 Co-Authored-By: Claude Opus 5.5 --- README.md | 8 +- docs/personal-learning.md | 92 ++++++++++++-------- neural/fixtures/personal.json | 14 ++- scripts/personal_evaluate.py | 16 +++- src/evaluation.rs | 2 +- src/lib.rs | 159 +++++++++++++++++++++++++--------- tests/prediction.rs | 75 ++++++++++++++-- 7 files changed, 273 insertions(+), 93 deletions(-) diff --git a/README.md b/README.md index 294442c..3d2f0bf 100644 --- a/README.md +++ b/README.md @@ -128,10 +128,10 @@ fn example() -> Result<()> { Fallible operations return typed `Error` values. Reuse a long-lived predictor; keep loading, imports, learning, refresh and resets on a worker thread. Prediction uses memory only. The model uses unigram/bigram/trigram weights **0.1/0.3/0.6**, -renormalized over available contexts, and combines baseline counts with **500×** -personal counts before probability calculation. Learned text is tiny beside the -baseline, so a lower weight leaves personal words almost never suggested; see -[personal learning](docs/personal-learning.md). Prefix matches rank by score, +renormalized over available contexts. Learned counts are kept apart from the +baseline and added to it when predicting, one learned occurrence counting as up to +**500** baseline ones, less as more is learned. A word the baseline lacks is +suggested once learned twice. See [personal learning](docs/personal-learning.md). Prefix matches rank by score, then alphabetically. Schema v1 and existing personal databases remain compatible. ## Validation and model selection diff --git a/docs/personal-learning.md b/docs/personal-learning.md index 06ef302..fd3a163 100644 --- a/docs/personal-learning.md +++ b/docs/personal-learning.md @@ -1,48 +1,72 @@ # Personal learning weight -Issue #29 raises the weight of learned counts from 5 to 500 baseline -occurrences each. Learned text is tiny beside the baseline. At 5×, a name or care -term typed a few times hardly ever reaches the suggestions. +Issue #29 makes learned words count. Learned counts used to be added to a copy +of the baseline at 5×. Learned text is tiny beside the baseline, so a name or +care term typed a few times hardly ever reached the suggestions. + +## Design + +Learned counts are kept apart from the baseline and combined when predicting: + +- **Weight.** One learned occurrence counts as 500 baseline occurrences while + little has been learned. The weight is `500 / (1 + learned words / 2000)`, so + it has halved after 2,000 learned words. However much is learned, learned + counts approach a bounded share of the baseline instead of outgrowing it. +- **New words.** A word the baseline does not know is suggested only once it + has been learned twice, so a single misspelling is not. Words the baseline + knows count from their first use. +- **Memory.** The baseline is no longer copied. Peak memory of the predictor + falls from 362 MiB to 134 MiB, with or without a personal database. +- **No learning.** With no personal database, or an empty one, predictions are + bit-identical to before. + +Scores are computed in floating point, so stored counts cannot overflow when +weighted. The schema and API are unchanged, and existing personal databases +keep working. The `personal_weight: 5` in the v1 model manifest records how that +model was described at build time; this document describes the runtime +weighting. ## Measurement `scripts/personal_evaluate.py` uses `neural/fixtures/personal.json`. The fixture -is synthetic text imitating one person's typing; it is not user data. It has 40 -learned sentences and 25 held-out test sentences that reuse their names, places, -care words and interests. Each test word is queried with 0–2 typed letters, 445 -queries in all. 78 of them are personal words: learned and rare in the baseline. -Both six slots count. - -| Personal weight | Six-slot top six | Personal words | -| ---: | ---: | ---: | -| no learning | 51.0% | 1.3% | -| 5 | 54.6% | 16.7% | -| 50 | 57.1% | 28.2% | -| **500** | **61.8%** | **53.8%** | -| 2,000 | 62.9% | 60.3% | -| 5,000 | 64.0% | 65.4% | +is synthetic text imitating one person's typing; it is not user data. + +- It has 44 learned sentences and 25 held-out test sentences that reuse their + names, places, care words and interests. Four of the learned sentences each + contain one misspelling. +- Each test word is queried with 0–2 typed letters: 445 queries. +- 78 queries are personal words: learned, and with fewer than 10 baseline + occurrences. They cover only 18 distinct words, each learned 1–3 times. +- A query is a hit when the word is in any of the six slots: 3 statistical and + 3 generated. + +Results from this branch, with v0.2.1 generation: + +| Weighting | Top six | Personal words | Misspelling suggested | +| --- | ---: | ---: | ---: | +| no learning | 50.3% | 1.3% | 0% | +| 500×, new words at once | 61.8% | 53.8% | 0.2% | +| **500× bounded, new words twice** | **60.4%** | **46.2%** | **0%** | + +Some personal words in the fixture are learned only once, so requiring two uses +costs part of the gain here. In real use, names that matter recur. + +An earlier sweep of a plain multiplier, measured on the faster generation build +from #26 and before the misspellings were added, showed how much the weight +matters. Personal words reached the six slots in 16.7% of queries at 5×, 28.2% +at 50×, 53.8% at 500×, 60.3% at 2,000× and 65.4% at 5,000×. General English must not suffer for someone who has learned text. 600 queries from the frozen general workload, with natural context, were run with that -person's learning present. "Light" is the 40 sentences learned once; "heavy" is -the same text learned ten times. - -| Personal weight | Light | Heavy | -| ---: | ---: | ---: | -| no learning | 71.0% | | -| 5 | 71.2% | 71.2% | -| **500** | **71.5%** | **71.3%** | -| 2,000 | 71.5% | 71.0% | -| 5,000 | 71.3% | 71.0% | - -Gains continue above 500 but slow, and the test sentences reuse exactly the -learned vocabulary, which favours high weights. 500 keeps most of the gain with -no measurable cost to general text. Differences of under about two points on 600 -queries are within noise. +person's learning present. "Light" is the 40 original sentences learned once; +"heavy" is the same text learned ten times. On this branch, with v0.2.1 +generation: no learning 70.3%, light 70.7%, heavy 70.2%. These differences are +within noise for 600 queries. The general-workload runs used a separate local +harness, not this script. Offering learned words to the neural generator as extra candidates was also -tried. It added under half a point at 500 and nearly doubled generation -latency, so it is not included. +tried. It added under half a point and nearly doubled generation latency, so it +is not included. Measured on Windows 11 with an AMD Ryzen AI 9 HX 370. These are synthetic regression measurements, not evidence of real users' accuracy. diff --git a/neural/fixtures/personal.json b/neural/fixtures/personal.json index 4b90a2c..6eaec95 100644 --- a/neural/fixtures/personal.json +++ b/neural/fixtures/personal.json @@ -1,5 +1,5 @@ { - "note": "Synthetic text written for this evaluation, imitating one person's typing with their own names, places, care words and interests. It is not real user data. `learn` is learned into a personal database; `test` is held out and reuses the same vocabulary in new sentences.", + "note": "Synthetic text written for this evaluation, imitating one person's typing with their own names, places, care words and interests. It is not real user data. `learn` is learned into a personal database; `test` is held out and reuses the same vocabulary in new sentences. Four learn sentences each contain one misspelling, listed in `misspellings`, to check that a word typed wrongly once is not suggested.", "learn": [ "Siobhan is coming at ten to help with the hoist.", "Can you ask Siobhan to bring my blue cardigan?", @@ -40,7 +40,11 @@ "Can you open the curtains please?", "I am too warm, can you turn the heating down?", "Thank you, that is much better.", - "I love you all very much." + "I love you all very much.", + "I hav a sore throat today.", + "Can you open teh window please?", + "I am tired becuase I slept badly.", + "Siobahn said she will be late." ], "test": [ "Is Siobhan working this weekend?", @@ -68,5 +72,11 @@ "Can you get the suction ready please?", "Aoife says the physio is cancelled.", "I am tired, can I go to bed?" + ], + "misspellings": [ + "hav", + "teh", + "becuase", + "siobahn" ] } diff --git a/scripts/personal_evaluate.py b/scripts/personal_evaluate.py index a431987..beeaca3 100644 --- a/scripts/personal_evaluate.py +++ b/scripts/personal_evaluate.py @@ -44,8 +44,10 @@ def run(args, personal, items): if personal: command += ['--personal', str(personal)] client = Client(command) - hits = dict(all=0, personal=0, generated=0) + hits = dict(all=0, personal=0, generated=0, misspelled=0) + misspellings = set(json.loads(FIXTURE.read_bytes())['misspellings']) times = [] + failures = 0 try: for before, prefix, target, own in items: client.process.stdin.write('{"command":"reset"}\n') @@ -56,23 +58,31 @@ def run(args, personal, items): prefix=prefix, session=1)) + '\n') client.process.stdin.flush() immediate = client.receive() + assert immediate['type'] == 'immediate', immediate['type'] instant = immediate['result']['words'] generated = [] if immediate['result']['refinement_requested']: reply = client.receive() if reply['type'] == 'refined': + assert reply['result']['request_id'] == immediate['result']['request_id'] generated = reply['result']['words'] times.append((time.perf_counter() - start) * 1000) + else: + failures += 1 + client.retry() hit = target in instant + generated hits['all'] += hit hits['personal'] += hit and own hits['generated'] += target in generated + hits['misspelled'] += bool(misspellings.intersection(instant + generated)) finally: client.close() count = len(items) own = sum(item[3] for item in items) return dict(top6=hits['all'] / count, personal_top6=hits['personal'] / own, - generated_hits=hits['generated'] / count, latency=timing(times)) + generated_hits=hits['generated'] / count, + misspelling_in_slots=hits['misspelled'] / count, + generation_failures=failures, latency=timing(times)) def main(): @@ -91,7 +101,7 @@ def main(): # Newlines separate context exactly as separate learn calls would. text = '\n'.join(json.loads(FIXTURE.read_bytes())['learn']) subprocess.run([str(args.learner), 'learn', '--baseline', str(args.baseline), - '--personal', str(personal)], input=text, text=True, + '--personal', str(personal)], input=text, text=True, encoding='utf-8', check=True, capture_output=True) report = dict( queries=len(items), personal_queries=sum(item[3] for item in items), diff --git a/src/evaluation.rs b/src/evaluation.rs index 46216f0..57aea88 100644 --- a/src/evaluation.rs +++ b/src/evaluation.rs @@ -179,7 +179,7 @@ pub fn score_database(path: &Path, text: &str, hardware: String) -> Result, - /// The personal database's `data_version` when `combined` last matched it. + /// The personal database's `data_version` when `learned` last matched it. /// It changes only when another connection commits. seen: Option, } @@ -421,8 +429,8 @@ impl Predictor { None }; let mut result = Self { - baseline: counts.clone(), - combined: counts, + baseline: counts, + learned: Counts::default(), personal, seen: None, }; @@ -432,13 +440,13 @@ impl Predictor { /// Refresh the in-memory snapshot after another process changes personal learning. /// On failure the previous prediction snapshot is retained. pub fn refresh_personal(&mut self) -> Result<()> { - let mut combined = self.baseline.clone(); + let mut learned = Counts::default(); if let Some(db) = &mut self.personal { let tx = db.transaction()?; if check(&tx)? != "personal" { return Err(Error::Invalid("expected personal database".into())); } - combined.merge(&load_checked(&tx, "personal")?, PERSONAL_WEIGHT); + learned = load_checked(&tx, "personal")?; let version = data_version(&tx)?; if let Err(error) = tx.commit() { // The outcome is unknown, so the next write reloads. @@ -447,9 +455,32 @@ impl Predictor { } self.seen = Some(version); } - self.combined = combined; + self.learned = learned; Ok(()) } + /// The current weight of one learned occurrence. + fn personal_weight(&self) -> f64 { + // Summed as floats: stored counts are bounded one by one, not in total. + let learned: f64 = self + .learned + .0 + .get(&Vec::new()) + .map_or(0., |w| w.values().map(|c| *c as f64).sum()); + PERSONAL_WEIGHT / (1. + learned / PERSONAL_SATURATION) + } + /// Whether learned counts of `word` are used: always for words the + /// baseline knows, and for others once learned `NEW_WORD_REPEATS` times. + fn known(&self, word: &str) -> bool { + let count = |counts: &Counts| { + counts + .0 + .get(&Vec::new()) + .and_then(|w| w.get(word)) + .copied() + .unwrap_or(0) + }; + count(&self.baseline) > 0 || count(&self.learned) >= NEW_WORD_REPEATS + } pub fn predict(&self, before: &str, prefix: &str, options: Options) -> Vec { let prefix = normalize(prefix); if prefix.graphemes(true).count() < options.min_chars || options.limit == 0 { @@ -460,35 +491,68 @@ impl Predictor { .next() .unwrap_or(""); let context = sentences(tail).pop().unwrap_or_default(); + let personal = self.personal_weight(); + let empty = BTreeMap::new(); let mut distributions = Vec::new(); for (n, weight) in [(0, 0.1), (1, 0.3), (2, 0.6)] { if n > context.len() || (options.unigram_only && n != 0) { continue; } - if let Some(words) = self.combined.0.get(&context[context.len() - n..]) { - let total: f64 = words.values().map(|c| *c as f64).sum(); - distributions.push((words, total, weight)); + let key = &context[context.len() - n..]; + let base = self.baseline.0.get(key); + let learned = self + .learned + .0 + .get(key) + .filter(|words| words.keys().any(|w| self.known(w))); + if base.is_none() && learned.is_none() { + continue; + } + let base = base.unwrap_or(&empty); + let mut total: f64 = base.values().map(|c| *c as f64).sum(); + if let Some(learned) = learned { + let own: f64 = learned + .iter() + .filter(|(w, _)| self.known(w)) + .map(|(_, c)| *c as f64) + .sum(); + total += personal * own; } + distributions.push((base, learned, total, weight)); } - let weight_sum: f64 = distributions.iter().map(|(_, _, w)| w).sum(); - let mut result = Vec::new(); - if let Some(vocabulary) = self.combined.0.get(&Vec::new()) { - for (word, _) in vocabulary.range(prefix.clone()..) { - if !word.starts_with(&prefix) { - break; - } + let weight_sum: f64 = distributions.iter().map(|(_, _, _, w)| w).sum(); + // Baseline words, and learned words once known, starting with the prefix. + let mut words: BTreeSet<&String> = BTreeSet::new(); + for counts in [&self.baseline, &self.learned] { + if let Some(vocabulary) = counts.0.get(&Vec::new()) { + words.extend( + vocabulary + .range(prefix.clone()..) + .take_while(|(w, _)| w.starts_with(&prefix)) + .map(|(w, _)| w) + .filter(|w| self.known(w)), + ); + } + } + let mut result: Vec = words + .into_iter() + .map(|word| { let score = distributions .iter() - .map(|(words, total, w)| { - words.get(word).copied().unwrap_or(0) as f64 / total * w / weight_sum + .map(|(base, learned, total, w)| { + let own = learned.and_then(|l| l.get(word)).copied().unwrap_or(0); + (base.get(word).copied().unwrap_or(0) as f64 + personal * own as f64) + / total + * w + / weight_sum }) .sum(); - result.push(Suggestion { + Suggestion { word: word.clone(), score, - }); - } - } + } + }) + .collect(); result.sort_unstable_by(|a, b| { b.score .total_cmp(&a.score) @@ -524,12 +588,10 @@ impl Predictor { // A connection's own commits leave its data_version unchanged. let version = data_version(&tx)?; let current = self.seen == Some(version) && standard_schema(&tx)?; - let combined = if current { + let reloaded = if current { None } else { - let mut combined = self.baseline.clone(); - combined.merge(&load_checked(&tx, "personal")?, PERSONAL_WEIGHT); - Some(combined) + Some(load_checked(&tx, "personal")?) }; if let Err(error) = tx.commit() { // The outcome is unknown, so the next write reloads. @@ -537,9 +599,9 @@ impl Predictor { return Err(error.into()); } self.seen = Some(version); - match combined { - Some(combined) => self.combined = combined, - None if imported => self.combined.merge(&delta, PERSONAL_WEIGHT), + match reloaded { + Some(learned) => self.learned = learned, + None if imported => self.learned.merge(&delta, 1), None => {} } Ok(imported) @@ -556,7 +618,6 @@ impl Predictor { .personal .as_mut() .ok_or_else(|| Error::Invalid("personal database required".into()))?; - let combined = self.baseline.clone(); let tx = db.transaction_with_behavior(TransactionBehavior::Immediate)?; if check(&tx)? != "personal" { return Err(Error::Invalid("expected personal database".into())); @@ -579,12 +640,12 @@ impl Predictor { return Err(error.into()); } self.seen = Some(version); - self.combined = combined; + self.learned = Counts::default(); Ok(()) } /// Approximate bytes of stored UTF-8 keys and counts, excluding allocator/tree overhead. pub fn model_payload_bytes(&self) -> usize { - [&self.baseline, &self.combined] + [&self.baseline, &self.learned] .iter() .map(|counts| { counts @@ -604,17 +665,33 @@ impl Predictor { mod tests { use super::*; - /// Adds a word to the baseline that only a full reload copies into the - /// snapshot. + /// Adds a learned count only memory has, which a reload from the + /// database removes and adding counts in place keeps. fn mark(p: &mut Predictor, word: &str) { - p.baseline + p.learned .0 .entry(vec![]) .or_default() .insert(word.into(), 1); } fn reloaded(p: &Predictor, word: &str) -> bool { - p.combined.0[&Vec::new()].contains_key(word) + !p.learned.0[&Vec::new()].contains_key(word) + } + + #[test] + fn the_learned_weight_halves_as_much_is_learned() { + let dir = tempfile::tempdir().unwrap(); + let base = dir.path().join("base.sqlite"); + let personal = dir.path().join("personal.sqlite"); + build(&base, "hello water", "fixture").unwrap(); + let mut p = Predictor::open(&base, Some(&personal)).unwrap(); + assert_eq!(p.personal_weight(), PERSONAL_WEIGHT); + p.learned + .0 + .entry(vec![]) + .or_default() + .insert("hello".into(), PERSONAL_SATURATION as i64); + assert_eq!(p.personal_weight(), PERSONAL_WEIGHT / 2.); } #[test] diff --git a/tests/prediction.rs b/tests/prediction.rs index 0dbadb7..688df31 100644 --- a/tests/prediction.rs +++ b/tests/prediction.rs @@ -85,12 +85,12 @@ fn learning_persistence_import_dedup_reset_and_baseline_immutability() { let original = fs::read(&path).unwrap(); let personal = tmp.path().join("personal.sqlite"); let text = tmp.path().join("training.txt"); - fs::write(&text, "I drink watermelon.").unwrap(); + fs::write(&text, "I drink watermelon. I like watermelon.").unwrap(); let mut p = Predictor::open(&path, Some(&personal)).unwrap(); assert!(p.import(&text).unwrap()); assert!(!p.import(&text).unwrap()); assert_eq!(words(&p, "I drink ", "wa")[0], "watermelon"); - p.learn("I drink walnut.").unwrap(); + p.learn("I drink walnut. I drink walnut.").unwrap(); drop(p); let mut p = Predictor::open(&path, Some(&personal)).unwrap(); assert!(words(&p, "I drink ", "wa").contains(&"walnut".to_string())); @@ -197,7 +197,7 @@ fn cli_build_predict_learn_validate() { .stdin .take() .unwrap() - .write_all(b"hello walnut") + .write_all(b"hello walnut. hello walnut") .unwrap(); assert!(child.wait().unwrap().success()); let output = Command::new(exe) @@ -240,8 +240,11 @@ fn interpolation_renormalizes_and_personal_counts_are_weighted_before_probabilit assert!((backed_off - (0.3 * 0.5 + 0.1 / 9.0) / 0.4).abs() < 1e-12); p.learn("i like tea.").unwrap(); let learned = p.predict("i like ", "tea", opts).remove(0).score; - // One learned occurrence counts as 500 baseline ones. - assert!((learned - (0.9 * 501.0 / 502.0 + 0.1 * 501.0 / 1509.0)).abs() < 1e-12); + // One learned occurrence counts as 500 baseline ones, less as more is + // learned: here three words. + let w = 500.0 / (1.0 + 3.0 / 2000.0); + let expected = 0.9 * (1.0 + w) / (2.0 + w) + 0.1 * (1.0 + w) / (9.0 + 3.0 * w); + assert!((learned - expected).abs() < 1e-12); let total: f64 = p.predict("i like ", "", opts).iter().map(|s| s.score).sum(); assert!((total - 1.0).abs() < 1e-12); } @@ -346,7 +349,7 @@ fn validation_failure_rolls_back_disk_and_keeps_prediction_snapshot() { assert_eq!(predictor.predict("hello", "wa", Options::default()), before); assert_eq!(validate(&personal).unwrap().ngrams, 0); db.execute_batch("DROP TRIGGER poison").unwrap(); - predictor.learn("hello walnut").unwrap(); + predictor.learn("hello walnut. hello walnut").unwrap(); assert_eq!( predictor.predict("hello", "wa", Options::default())[0].word, "walnut" @@ -361,7 +364,7 @@ fn multiple_predictors_refresh_duplicate_imports_and_reset() { let personal = dir.path().join("personal.sqlite"); let training = dir.path().join("fixture.txt"); build(&base, "hello water", "fixture").unwrap(); - fs::write(&training, "hello walnut").unwrap(); + fs::write(&training, "hello walnut. hello walnut").unwrap(); let mut first = Predictor::open(&base, Some(&personal)).unwrap(); let mut second = Predictor::open(&base, Some(&personal)).unwrap(); assert!(first.import(&training).unwrap()); @@ -487,7 +490,9 @@ fn learning_after_another_writer_includes_its_counts() { first .learn("I drink walnut milk. I drink walnut milk.") .unwrap(); - second.learn("I drink walrus milk.").unwrap(); + second + .learn("I drink walrus milk. I drink walrus milk.") + .unwrap(); assert!(words(&second, "I drink", "wal").starts_with(&["walnut".into(), "walrus".into()])); assert_matches_reload(&second, &base, &personal); second.learn("We walk home.").unwrap(); @@ -571,3 +576,57 @@ fn a_replacing_constraint_is_still_validated_before_commit() { assert!(p.learn("say hello").is_err()); assert!(Predictor::open(&base, Some(&personal)).is_ok()); } + +#[test] +fn a_misspelling_learned_once_is_not_suggested_but_a_new_word_used_twice_is() { + let (tmp, base) = fixture(); + let personal = tmp.path().join("personal.sqlite"); + let mut p = Predictor::open(&base, Some(&personal)).unwrap(); + p.learn("I drink teh water.").unwrap(); + assert!(!words(&p, "I drink ", "t").contains(&"teh".to_string())); + assert!(!words(&p, "", "te").contains(&"teh".to_string())); + // A word the baseline knows counts at once. + p.learn("We walk apple.").unwrap(); + assert_eq!(words(&p, "We walk ", "a")[0], "apple"); + p.learn("I drink teh tea.").unwrap(); + assert!(words(&p, "I drink ", "t").contains(&"teh".to_string())); +} + +#[test] +fn an_empty_personal_database_predicts_exactly_like_none() { + let (tmp, base) = fixture(); + let personal = tmp.path().join("personal.sqlite"); + let with = Predictor::open(&base, Some(&personal)).unwrap(); + let without = Predictor::open(&base, None).unwrap(); + for (context, prefix) in [("", ""), ("I drink", ""), ("I drink", "wa"), ("we", "w")] { + assert_eq!( + ranked(&with, context, prefix), + ranked(&without, context, prefix) + ); + } +} + +#[test] +fn huge_stored_personal_counts_do_not_overflow() { + let (tmp, base) = fixture(); + let personal = tmp.path().join("personal.sqlite"); + let mut p = Predictor::open(&base, Some(&personal)).unwrap(); + p.learn("I drink water. I drink tea.").unwrap(); + drop(p); + let db = rusqlite::Connection::open(&personal).unwrap(); + db.execute("UPDATE counts SET count = 922337203685477580", []) + .unwrap(); + validate(&personal).unwrap(); + let p = Predictor::open(&base, Some(&personal)).unwrap(); + let ranked = p.predict( + "I drink ", + "", + Options { + limit: 20, + min_chars: 0, + unigram_only: false, + }, + ); + assert!(!ranked.is_empty()); + assert!(ranked.iter().all(|s| s.score.is_finite())); +} From 09bb4e79b2367b8bdca96bad4c30393a34903c0a Mon Sep 17 00:00:00 2001 From: Owen McGirr Date: Mon, 12 Oct 2026 15:52:11 +0100 Subject: [PATCH 3/8] Cache usable learned counts and guard rare baseline words Prediction was about twice as slow because every candidate and learned entry was checked on each call. What prediction uses of the learned counts, with per-context totals and the current weight, is now rebuilt only when they change: an empty-sentence prediction takes 5.3 ms against 7.5 ms before. The baseline itself holds thousands of rare misspellings, so learned counts of a word are used once it is learned twice unless the baseline has it at least three times. Learning "hav" once no longer suggests it. Adds an exact bit-for-bit check of the no-learning formula and a timing test. Refs #29 Co-Authored-By: Claude Opus 5.5 --- README.md | 7 ++- docs/personal-learning.md | 22 +++++-- docs/production.md | 5 +- src/lib.rs | 117 +++++++++++++++++++++----------------- tests/prediction.rs | 68 ++++++++++++++++++++-- 5 files changed, 151 insertions(+), 68 deletions(-) diff --git a/README.md b/README.md index 3d2f0bf..18fcf15 100644 --- a/README.md +++ b/README.md @@ -130,9 +130,10 @@ keep loading, imports, learning, refresh and resets on a worker thread. Predicti uses memory only. The model uses unigram/bigram/trigram weights **0.1/0.3/0.6**, renormalized over available contexts. Learned counts are kept apart from the baseline and added to it when predicting, one learned occurrence counting as up to -**500** baseline ones, less as more is learned. A word the baseline lacks is -suggested once learned twice. See [personal learning](docs/personal-learning.md). Prefix matches rank by score, -then alphabetically. Schema v1 and existing personal databases remain compatible. +**500** baseline ones, less as more is learned. A word the baseline lacks, or has +seen fewer than three times, counts once learned twice. See +[personal learning](docs/personal-learning.md). Prefix matches rank by score, then +alphabetically. Schema v1 and existing personal databases remain compatible. ## Validation and model selection diff --git a/docs/personal-learning.md b/docs/personal-learning.md index fd3a163..3b80ebb 100644 --- a/docs/personal-learning.md +++ b/docs/personal-learning.md @@ -11,12 +11,22 @@ Learned counts are kept apart from the baseline and combined when predicting: - **Weight.** One learned occurrence counts as 500 baseline occurrences while little has been learned. The weight is `500 / (1 + learned words / 2000)`, so it has halved after 2,000 learned words. However much is learned, learned - counts approach a bounded share of the baseline instead of outgrowing it. -- **New words.** A word the baseline does not know is suggested only once it - has been learned twice, so a single misspelling is not. Words the baseline - knows count from their first use. -- **Memory.** The baseline is no longer copied. Peak memory of the predictor - falls from 362 MiB to 134 MiB, with or without a personal database. + unigram mass approaches 500 × 2,000 = 1,000,000 occurrences, about the size + of the production baseline's 1,076,617, so learned words approach half the + unigram distribution rather than outgrowing the baseline. These constants are + tuned to this baseline; a much larger or smaller one would change that share. +- **New and rare words.** Learned counts of a word are used once it has been + learned twice, unless the baseline has it at least three times. A single + misspelling is not suggested, including one the baseline itself contains + once or twice: the production baseline has over 9,000 such words, many of + them misspellings. Words the baseline knows well count from their first use. + A word learned once still counts as context for the word after it. +- **Memory and speed.** The baseline is no longer copied. Peak memory of the + predictor falls from 362 MiB to 134 MiB, with or without a personal database. + What prediction uses of the learned counts is rebuilt when they change, so a + prediction costs about the same as before: 5.3 ms instead of 7.5 ms for an + empty sentence on the production baseline, and 8.9 ms after 2,000 new words + were learned twice. - **No learning.** With no personal database, or an empty one, predictions are bit-identical to before. diff --git a/docs/production.md b/docs/production.md index bcce03d..7ab4f0f 100644 --- a/docs/production.md +++ b/docs/production.md @@ -49,8 +49,9 @@ a noisy shared-runner gate. Dependency audit warnings also fail CI. Open one long-lived `Predictor` on a worker thread. Reusing it keeps prediction entirely in memory; spawning a CLI process or reopening the model for each keystroke incurs cold validation/loading. Loading the promoted model takes about -1.6 seconds on the development M2 Max, and learning validates personal state and -copies model counts. Keep open/import/learn/reset/refresh off the UI thread. +1.6 seconds on the development M2 Max. Learning writes only learned counts and +usually takes milliseconds; it reloads the personal database when another +connection has changed it. Keep open/import/learn/reset/refresh off the UI thread. `Predictor` is an owned, mutable learning session. Serialize access in the host application; do not share an unsynchronized connection across threads. Multiple diff --git a/src/lib.rs b/src/lib.rs index dfacf59..70ea0be 100644 --- a/src/lib.rs +++ b/src/lib.rs @@ -353,19 +353,32 @@ const PERSONAL_WEIGHT: f64 = 500.; /// learned counts approach a bounded share of the baseline rather than /// outgrowing it. const PERSONAL_SATURATION: f64 = 2000.; -/// Times a word the baseline lacks must be learned before it is suggested, so -/// a single misspelling is not. +/// Times a word must be learned before its learned counts are used, unless the +/// baseline has it at least `TRUSTED_BASELINE` times. A single misspelling, +/// including one the baseline itself has once or twice, is not suggested. const NEW_WORD_REPEATS: i64 = 2; +const TRUSTED_BASELINE: i64 = 3; pub struct Predictor { baseline: Counts, - /// Learned counts, combined with the baseline when predicting. + /// Learned counts, as stored. learned: Counts, + /// What predicting uses of `learned`, rebuilt whenever it changes. + usable: Usable, personal: Option, /// The personal database's `data_version` when `learned` last matched it. /// It changes only when another connection commits. seen: Option, } +/// The learned counts whose words may count, their total per context, and +/// the weight of one learned occurrence. +#[derive(Default)] +struct Usable { + counts: Counts, + totals: BTreeMap, f64>, + weight: f64, +} + fn data_version(db: &Connection) -> Result { Ok(db.query_row("PRAGMA data_version", [], |r| r.get(0))?) } @@ -431,6 +444,7 @@ impl Predictor { let mut result = Self { baseline: counts, learned: Counts::default(), + usable: Usable::default(), personal, seen: None, }; @@ -456,30 +470,36 @@ impl Predictor { self.seen = Some(version); } self.learned = learned; + self.rebuild(); Ok(()) } - /// The current weight of one learned occurrence. - fn personal_weight(&self) -> f64 { - // Summed as floats: stored counts are bounded one by one, not in total. - let learned: f64 = self - .learned - .0 - .get(&Vec::new()) - .map_or(0., |w| w.values().map(|c| *c as f64).sum()); - PERSONAL_WEIGHT / (1. + learned / PERSONAL_SATURATION) - } - /// Whether learned counts of `word` are used: always for words the - /// baseline knows, and for others once learned `NEW_WORD_REPEATS` times. - fn known(&self, word: &str) -> bool { - let count = |counts: &Counts| { - counts - .0 - .get(&Vec::new()) - .and_then(|w| w.get(word)) - .copied() - .unwrap_or(0) + /// Rebuilds what predicting uses after `learned` changes. Summed as + /// floats: stored counts are bounded one by one, not in total. + fn rebuild(&mut self) { + let empty = BTreeMap::new(); + let learned = self.learned.0.get(&Vec::new()).unwrap_or(&empty); + let baseline = self.baseline.0.get(&Vec::new()).unwrap_or(&empty); + let counts = |word: &String| { + baseline.get(word).copied().unwrap_or(0) >= TRUSTED_BASELINE + || learned.get(word).copied().unwrap_or(0) >= NEW_WORD_REPEATS }; - count(&self.baseline) > 0 || count(&self.learned) >= NEW_WORD_REPEATS + let mut usable = Usable::default(); + for (context, words) in &self.learned.0 { + let kept: BTreeMap = words + .iter() + .filter(|(w, _)| counts(w)) + .map(|(w, c)| (w.clone(), *c)) + .collect(); + if !kept.is_empty() { + usable + .totals + .insert(context.clone(), kept.values().map(|c| *c as f64).sum()); + usable.counts.0.insert(context.clone(), kept); + } + } + let learned: f64 = learned.values().map(|c| *c as f64).sum(); + usable.weight = PERSONAL_WEIGHT / (1. + learned / PERSONAL_SATURATION); + self.usable = usable; } pub fn predict(&self, before: &str, prefix: &str, options: Options) -> Vec { let prefix = normalize(prefix); @@ -491,7 +511,7 @@ impl Predictor { .next() .unwrap_or(""); let context = sentences(tail).pop().unwrap_or_default(); - let personal = self.personal_weight(); + let personal = self.usable.weight; let empty = BTreeMap::new(); let mut distributions = Vec::new(); for (n, weight) in [(0, 0.1), (1, 0.3), (2, 0.6)] { @@ -500,40 +520,32 @@ impl Predictor { } let key = &context[context.len() - n..]; let base = self.baseline.0.get(key); - let learned = self - .learned - .0 - .get(key) - .filter(|words| words.keys().any(|w| self.known(w))); + let learned = self.usable.counts.0.get(key); if base.is_none() && learned.is_none() { continue; } let base = base.unwrap_or(&empty); let mut total: f64 = base.values().map(|c| *c as f64).sum(); - if let Some(learned) = learned { - let own: f64 = learned - .iter() - .filter(|(w, _)| self.known(w)) - .map(|(_, c)| *c as f64) - .sum(); - total += personal * own; + if learned.is_some() { + total += personal * self.usable.totals[key]; } distributions.push((base, learned, total, weight)); } let weight_sum: f64 = distributions.iter().map(|(_, _, _, w)| w).sum(); - // Baseline words, and learned words once known, starting with the prefix. - let mut words: BTreeSet<&String> = BTreeSet::new(); - for counts in [&self.baseline, &self.learned] { - if let Some(vocabulary) = counts.0.get(&Vec::new()) { - words.extend( - vocabulary - .range(prefix.clone()..) - .take_while(|(w, _)| w.starts_with(&prefix)) - .map(|(w, _)| w) - .filter(|w| self.known(w)), - ); - } + // Baseline words, then learned words the baseline lacks, starting with + // the prefix. + fn range<'a>(counts: &'a Counts, prefix: &str) -> impl Iterator { + counts + .0 + .get(&Vec::new()) + .into_iter() + .flat_map(move |v| v.range(prefix.to_owned()..)) + .take_while(move |(w, _)| w.starts_with(prefix)) + .map(|(w, _)| w) } + let baseline = self.baseline.0.get(&Vec::new()).unwrap_or(&empty); + let mut words: Vec<&String> = range(&self.baseline, &prefix).collect(); + words.extend(range(&self.usable.counts, &prefix).filter(|w| !baseline.contains_key(*w))); let mut result: Vec = words .into_iter() .map(|word| { @@ -604,6 +616,7 @@ impl Predictor { None if imported => self.learned.merge(&delta, 1), None => {} } + self.rebuild(); Ok(imported) } pub fn learn(&mut self, completed_text: &str) -> Result<()> { @@ -641,6 +654,7 @@ impl Predictor { } self.seen = Some(version); self.learned = Counts::default(); + self.rebuild(); Ok(()) } /// Approximate bytes of stored UTF-8 keys and counts, excluding allocator/tree overhead. @@ -685,13 +699,14 @@ mod tests { let personal = dir.path().join("personal.sqlite"); build(&base, "hello water", "fixture").unwrap(); let mut p = Predictor::open(&base, Some(&personal)).unwrap(); - assert_eq!(p.personal_weight(), PERSONAL_WEIGHT); + assert_eq!(p.usable.weight, PERSONAL_WEIGHT); p.learned .0 .entry(vec![]) .or_default() .insert("hello".into(), PERSONAL_SATURATION as i64); - assert_eq!(p.personal_weight(), PERSONAL_WEIGHT / 2.); + p.rebuild(); + assert_eq!(p.usable.weight, PERSONAL_WEIGHT / 2.); } #[test] diff --git a/tests/prediction.rs b/tests/prediction.rs index 688df31..e104b2a 100644 --- a/tests/prediction.rs +++ b/tests/prediction.rs @@ -238,12 +238,16 @@ fn interpolation_renormalizes_and_personal_counts_are_weighted_before_probabilit assert!((tea - (0.6 * 0.5 + 0.3 * 0.5 + 0.1 / 9.0)).abs() < 1e-12); let backed_off = p.predict("unknown like ", "tea", opts).remove(0).score; assert!((backed_off - (0.3 * 0.5 + 0.1 / 9.0) / 0.4).abs() < 1e-12); + // Words this rare in the baseline count once learned twice. + p.learn("i like tea.").unwrap(); + assert_eq!(p.predict("i like ", "tea", opts).remove(0).score, tea); p.learn("i like tea.").unwrap(); let learned = p.predict("i like ", "tea", opts).remove(0).score; // One learned occurrence counts as 500 baseline ones, less as more is - // learned: here three words. - let w = 500.0 / (1.0 + 3.0 / 2000.0); - let expected = 0.9 * (1.0 + w) / (2.0 + w) + 0.1 * (1.0 + w) / (9.0 + 3.0 * w); + // learned: here six words. + let w = 500.0 / (1.0 + 6.0 / 2000.0); + let expected = + 0.9 * (1.0 + 2.0 * w) / (2.0 + 2.0 * w) + 0.1 * (1.0 + 2.0 * w) / (9.0 + 6.0 * w); assert!((learned - expected).abs() < 1e-12); let total: f64 = p.predict("i like ", "", opts).iter().map(|s| s.score).sum(); assert!((total - 1.0).abs() < 1e-12); @@ -585,9 +589,13 @@ fn a_misspelling_learned_once_is_not_suggested_but_a_new_word_used_twice_is() { p.learn("I drink teh water.").unwrap(); assert!(!words(&p, "I drink ", "t").contains(&"teh".to_string())); assert!(!words(&p, "", "te").contains(&"teh".to_string())); - // A word the baseline knows counts at once. - p.learn("We walk apple.").unwrap(); - assert_eq!(words(&p, "We walk ", "a")[0], "apple"); + // A word the baseline knows well counts at once; one it has rarely seen, + // which may itself be a misspelling, needs a second use. + assert_eq!(words(&p, "We walk ", "")[0], "home"); + p.learn("We walk wine.").unwrap(); + assert_eq!(words(&p, "We walk ", "")[0], "home"); + p.learn("We walk drink.").unwrap(); + assert_eq!(words(&p, "We walk ", "")[0], "drink"); p.learn("I drink teh tea.").unwrap(); assert!(words(&p, "I drink ", "t").contains(&"teh".to_string())); } @@ -630,3 +638,51 @@ fn huge_stored_personal_counts_do_not_overflow() { assert!(!ranked.is_empty()); assert!(ranked.iter().all(|s| s.score.is_finite())); } + +#[test] +fn without_learning_scores_are_exactly_the_interpolated_counts() { + let (_tmp, base) = fixture(); + let p = Predictor::open(&base, None).unwrap(); + let water = p.predict("I drink ", "wa", Options::default()).remove(0); + assert_eq!(water.word, "water"); + // Seventeen baseline words; "i drink" and "drink" are followed by water + // twice and wine once. Summed in the order the predictor sums them. + let weights: f64 = 0.1 + 0.3 + 0.6; + let expected = + 0.0 + 2.0 / 17.0 * 0.1 / weights + 2.0 / 3.0 * 0.3 / weights + 2.0 / 3.0 * 0.6 / weights; + assert_eq!(water.score.to_bits(), expected.to_bits()); +} + +/// Run with SWITCHIFY_BASELINE pointing at the production english.sqlite. +#[test] +#[ignore = "requires explicit SWITCHIFY_BASELINE"] +fn prediction_cost_on_production_baseline() { + let base = std::path::PathBuf::from(std::env::var_os("SWITCHIFY_BASELINE").unwrap()); + let tmp = tempfile::tempdir().unwrap(); + let personal = tmp.path().join("personal.sqlite"); + let mut p = Predictor::open(&base, Some(&personal)).unwrap(); + let options = Options { + limit: 6, + min_chars: 0, + unigram_only: false, + }; + let time = |p: &Predictor, label: &str| { + for (before, prefix) in [("", ""), ("i want to", ""), ("the", "s")] { + let started = std::time::Instant::now(); + for _ in 0..50 { + p.predict(before, prefix, options); + } + println!( + "{label} {before:?} {prefix:?}: {:?}", + started.elapsed() / 50 + ); + } + }; + time(&p, "none"); + let text: String = (0..2000) + .map(|i| format!("I need to call friend{i} about the physio. ")) + .collect(); + p.learn(&text).unwrap(); + p.learn(&text).unwrap(); + time(&p, "learned"); +} From 5f7159a91fe3b6f9190248a946f11e7dc3d63806 Mon Sep 17 00:00:00 2001 From: Owen McGirr Date: Mon, 12 Oct 2026 16:30:49 +0100 Subject: [PATCH 4/8] Update learned counts in place and correct the speed claims Rebuilding every usable learned count on each learn cost about 350 ms at 40,000 learned words. A learned sentence now adds its usable counts in place, and a word reaching its second use adds its existing counts in one pass; counts are whole numbers, so totals equal a full rebuild, which a test checks. One learn at 40,000 learned words takes 190-270 ms against 180 ms before #30. The timing test now learns letter-only words, since digits split words, and the docs state measured costs instead of a claimed speedup. Refs #29 Co-Authored-By: Claude Opus 5.5 --- docs/personal-learning.md | 21 ++++-- docs/production.md | 5 +- src/lib.rs | 155 ++++++++++++++++++++++++++++++++------ tests/prediction.rs | 29 +++++-- 4 files changed, 172 insertions(+), 38 deletions(-) diff --git a/docs/personal-learning.md b/docs/personal-learning.md index 3b80ebb..47acafc 100644 --- a/docs/personal-learning.md +++ b/docs/personal-learning.md @@ -21,12 +21,19 @@ Learned counts are kept apart from the baseline and combined when predicting: once or twice: the production baseline has over 9,000 such words, many of them misspellings. Words the baseline knows well count from their first use. A word learned once still counts as context for the word after it. -- **Memory and speed.** The baseline is no longer copied. Peak memory of the - predictor falls from 362 MiB to 134 MiB, with or without a personal database. - What prediction uses of the learned counts is rebuilt when they change, so a - prediction costs about the same as before: 5.3 ms instead of 7.5 ms for an - empty sentence on the production baseline, and 8.9 ms after 2,000 new words - were learned twice. +- **Memory.** The baseline is no longer copied. Peak memory of the predictor + falls from 362 MiB to 134 MiB, with or without a personal database. A second, + filtered copy of the usable learned counts is kept for prediction; it is + small beside the baseline. +- **Speed.** What prediction uses of the learned counts, with their totals per + context, is updated in place when a sentence is learned and rebuilt only on + reload. The ignored test `prediction_cost_on_production_baseline` measures a + release build on the production baseline. Without learning, prediction costs + the same as #28: 5–8 ms for an empty sentence and about 1 ms after a typed + letter. With 2,000 new words learned it is 9–14 ms. With 40,000 new words, an + extreme case, it is 30–40 ms, about 1.5× #28. One learn takes 17–19 ms at + 2,000 new words and 190–270 ms at 40,000, against 15 ms and 180 ms for #28. + Most of that is the existing database check. - **No learning.** With no personal database, or an empty one, predictions are bit-identical to before. @@ -50,7 +57,7 @@ is synthetic text imitating one person's typing; it is not user data. - A query is a hit when the word is in any of the six slots: 3 statistical and 3 generated. -Results from this branch, with v0.2.1 generation: +Results from this branch's final commit, with v0.2.1 generation: | Weighting | Top six | Personal words | Misspelling suggested | | --- | ---: | ---: | ---: | diff --git a/docs/production.md b/docs/production.md index 7ab4f0f..9bdf999 100644 --- a/docs/production.md +++ b/docs/production.md @@ -50,8 +50,9 @@ Open one long-lived `Predictor` on a worker thread. Reusing it keeps prediction entirely in memory; spawning a CLI process or reopening the model for each keystroke incurs cold validation/loading. Loading the promoted model takes about 1.6 seconds on the development M2 Max. Learning writes only learned counts and -usually takes milliseconds; it reloads the personal database when another -connection has changed it. Keep open/import/learn/reset/refresh off the UI thread. +validates the personal database. With a few thousand learned words a sentence +takes tens of milliseconds, and a few hundred at tens of thousands. It reloads +the personal database when another connection has changed it. Keep open/import/learn/reset/refresh off the UI thread. `Predictor` is an owned, mutable learning session. Serialize access in the host application; do not share an unsynchronized connection across threads. Multiple diff --git a/src/lib.rs b/src/lib.rs index 70ea0be..70075c2 100644 --- a/src/lib.rs +++ b/src/lib.rs @@ -376,6 +376,8 @@ pub struct Predictor { struct Usable { counts: Counts, totals: BTreeMap, f64>, + /// All learned unigram occurrences, which set the weight. + learned: f64, weight: f64, } @@ -476,30 +478,104 @@ impl Predictor { /// Rebuilds what predicting uses after `learned` changes. Summed as /// floats: stored counts are bounded one by one, not in total. fn rebuild(&mut self) { + self.usable = Usable::default(); + let contexts: Vec> = self.learned.0.keys().cloned().collect(); + for context in &contexts { + self.rebuild_context(context); + } + self.usable.learned = self + .learned + .0 + .get(&Vec::new()) + .map_or(0., |w| w.values().map(|c| *c as f64).sum()); + self.reweigh(); + } + /// Updates what prediction uses after `delta` was added to `learned`: + /// its usable counts are added, and a word that has just become usable + /// gets all its learned counts. Counts are whole numbers, so adding them + /// in place gives the same totals as rebuilding. + fn update(&mut self, delta: &Counts) { let empty = BTreeMap::new(); + let added = delta.0.get(&Vec::new()).unwrap_or(&empty); let learned = self.learned.0.get(&Vec::new()).unwrap_or(&empty); - let baseline = self.baseline.0.get(&Vec::new()).unwrap_or(&empty); - let counts = |word: &String| { - baseline.get(word).copied().unwrap_or(0) >= TRUSTED_BASELINE - || learned.get(word).copied().unwrap_or(0) >= NEW_WORD_REPEATS - }; - let mut usable = Usable::default(); - for (context, words) in &self.learned.0 { - let kept: BTreeMap = words - .iter() - .filter(|(w, _)| counts(w)) - .map(|(w, c)| (w.clone(), *c)) - .collect(); - if !kept.is_empty() { - usable - .totals - .insert(context.clone(), kept.values().map(|c| *c as f64).sum()); - usable.counts.0.insert(context.clone(), kept); + let newly: BTreeSet<&String> = added + .iter() + .filter(|(word, count)| { + let now = learned.get(*word).copied().unwrap_or(0); + !self.trusted(word) && now >= NEW_WORD_REPEATS && now - *count < NEW_WORD_REPEATS + }) + .map(|(word, _)| word) + .collect(); + let mut changes: Vec<(Vec, String, i64)> = Vec::new(); + if !newly.is_empty() { + for (context, words) in &self.learned.0 { + for (word, count) in words { + if newly.contains(word) { + changes.push((context.clone(), word.clone(), *count)); + } + } } } - let learned: f64 = learned.values().map(|c| *c as f64).sum(); - usable.weight = PERSONAL_WEIGHT / (1. + learned / PERSONAL_SATURATION); - self.usable = usable; + for (context, words) in &delta.0 { + for (word, count) in words { + if !newly.contains(word) && self.usable_word(word) { + changes.push((context.clone(), word.clone(), *count)); + } + } + } + for (context, word, count) in changes { + *self.usable.totals.entry(context.clone()).or_default() += count as f64; + *self + .usable + .counts + .0 + .entry(context) + .or_default() + .entry(word) + .or_default() += count; + } + self.usable.learned += added.values().map(|c| *c as f64).sum::(); + self.reweigh(); + } + fn trusted(&self, word: &str) -> bool { + self.baseline + .0 + .get(&Vec::new()) + .and_then(|w| w.get(word)) + .is_some_and(|c| *c >= TRUSTED_BASELINE) + } + /// Whether learned counts of `word` are used. + fn usable_word(&self, word: &str) -> bool { + self.trusted(word) + || self + .learned + .0 + .get(&Vec::new()) + .and_then(|w| w.get(word)) + .is_some_and(|c| *c >= NEW_WORD_REPEATS) + } + fn rebuild_context(&mut self, context: &[String]) { + let kept: BTreeMap = self + .learned + .0 + .get(context) + .into_iter() + .flatten() + .filter(|(w, _)| self.usable_word(w)) + .map(|(w, c)| (w.clone(), *c)) + .collect(); + if kept.is_empty() { + self.usable.totals.remove(context); + self.usable.counts.0.remove(context); + } else { + self.usable + .totals + .insert(context.to_vec(), kept.values().map(|c| *c as f64).sum()); + self.usable.counts.0.insert(context.to_vec(), kept); + } + } + fn reweigh(&mut self) { + self.usable.weight = PERSONAL_WEIGHT / (1. + self.usable.learned / PERSONAL_SATURATION); } pub fn predict(&self, before: &str, prefix: &str, options: Options) -> Vec { let prefix = normalize(prefix); @@ -612,11 +688,16 @@ impl Predictor { } self.seen = Some(version); match reloaded { - Some(learned) => self.learned = learned, - None if imported => self.learned.merge(&delta, 1), + Some(learned) => { + self.learned = learned; + self.rebuild(); + } + None if imported => { + self.learned.merge(&delta, 1); + self.update(&delta); + } None => {} } - self.rebuild(); Ok(imported) } pub fn learn(&mut self, completed_text: &str) -> Result<()> { @@ -692,6 +773,34 @@ mod tests { !p.learned.0[&Vec::new()].contains_key(word) } + #[test] + fn updating_in_place_matches_rebuilding() { + let dir = tempfile::tempdir().unwrap(); + let base = dir.path().join("base.sqlite"); + let personal = dir.path().join("personal.sqlite"); + build(&base, "hello water. i like tea. we like tea.", "fixture").unwrap(); + let mut p = Predictor::open(&base, Some(&personal)).unwrap(); + for text in [ + "Siobhan likes zed water.", + "zed Siobhan likes tea.", + "hello zed. Siobhan said hello.", + "", + "we like Siobhan and zed", + ] { + p.learn(text).unwrap(); + let (counts, totals, weight) = ( + p.usable.counts.0.clone(), + p.usable.totals.clone(), + p.usable.weight, + ); + p.rebuild(); + assert_eq!(counts, p.usable.counts.0, "{text}"); + assert_eq!(totals, p.usable.totals, "{text}"); + assert_eq!(weight.to_bits(), p.usable.weight.to_bits(), "{text}"); + } + assert!(p.usable.counts.0[&Vec::new()].contains_key("siobhan")); + } + #[test] fn the_learned_weight_halves_as_much_is_learned() { let dir = tempfile::tempdir().unwrap(); diff --git a/tests/prediction.rs b/tests/prediction.rs index e104b2a..7824c2c 100644 --- a/tests/prediction.rs +++ b/tests/prediction.rs @@ -679,10 +679,27 @@ fn prediction_cost_on_production_baseline() { } }; time(&p, "none"); - let text: String = (0..2000) - .map(|i| format!("I need to call friend{i} about the physio. ")) - .collect(); - p.learn(&text).unwrap(); - p.learn(&text).unwrap(); - time(&p, "learned"); + // Digits split words, so each new word is spelled in letters. + let letters = |mut i: usize| { + let mut word = String::from("zq"); + loop { + word.push((b'a' + (i % 26) as u8) as char); + i /= 26; + if i == 0 { + return word; + } + } + }; + for (count, label) in [(2_000, "2k new words"), (40_000, "40k new words")] { + let text: String = (0..count) + .map(|i| format!("I need to call {} about the physio. ", letters(i))) + .collect(); + p.learn(&text).unwrap(); + p.learn(&text).unwrap(); + let started = std::time::Instant::now(); + p.learn("I need to call the physio about my hoist.") + .unwrap(); + println!("{label}: one learn {:?}", started.elapsed()); + time(&p, label); + } } From 6b80bbfe23493efe3affa72aeb72700b6b29c50f Mon Sep 17 00:00:00 2001 From: Owen McGirr Date: Mon, 12 Oct 2026 16:38:33 +0100 Subject: [PATCH 5/8] Test trusted and imported updates and state measured parity The in-place update test now covers words the baseline trusts, an import and a duplicate import. The timing test also times a learn that makes a new word usable. Measured against #28 on the same workload, prediction and learning costs match within noise, so the docs say so. Refs #29 Co-Authored-By: Claude Opus 5.5 --- docs/personal-learning.md | 14 +++---- docs/production.md | 4 +- src/lib.rs | 79 ++++++++++++++++++++++----------------- tests/prediction.rs | 10 ++++- 4 files changed, 62 insertions(+), 45 deletions(-) diff --git a/docs/personal-learning.md b/docs/personal-learning.md index 47acafc..66a00fb 100644 --- a/docs/personal-learning.md +++ b/docs/personal-learning.md @@ -27,13 +27,13 @@ Learned counts are kept apart from the baseline and combined when predicting: small beside the baseline. - **Speed.** What prediction uses of the learned counts, with their totals per context, is updated in place when a sentence is learned and rebuilt only on - reload. The ignored test `prediction_cost_on_production_baseline` measures a - release build on the production baseline. Without learning, prediction costs - the same as #28: 5–8 ms for an empty sentence and about 1 ms after a typed - letter. With 2,000 new words learned it is 9–14 ms. With 40,000 new words, an - extreme case, it is 30–40 ms, about 1.5× #28. One learn takes 17–19 ms at - 2,000 new words and 190–270 ms at 40,000, against 15 ms and 180 ms for #28. - Most of that is the existing database check. + reload. Measured by the ignored test `prediction_cost_on_production_baseline` + in a release build on the production baseline, costs match #28 within noise: + about 5 ms for an empty sentence and 1 ms after a typed letter without + learning, 9–15 ms with 2,000 new words learned, and 30–40 ms with 40,000, an + extreme case. One learn takes 17–23 ms at 2,000 new words and about 190 ms + at 40,000, including one that makes a new word usable. Most of that is the + existing database check. - **No learning.** With no personal database, or an empty one, predictions are bit-identical to before. diff --git a/docs/production.md b/docs/production.md index 9bdf999..5600402 100644 --- a/docs/production.md +++ b/docs/production.md @@ -50,8 +50,8 @@ Open one long-lived `Predictor` on a worker thread. Reusing it keeps prediction entirely in memory; spawning a CLI process or reopening the model for each keystroke incurs cold validation/loading. Loading the promoted model takes about 1.6 seconds on the development M2 Max. Learning writes only learned counts and -validates the personal database. With a few thousand learned words a sentence -takes tens of milliseconds, and a few hundred at tens of thousands. It reloads +validates the personal database: with a few thousand learned words a sentence +takes tens of milliseconds, and about 200 ms at tens of thousands. It reloads the personal database when another connection has changed it. Keep open/import/learn/reset/refresh off the UI thread. `Predictor` is an owned, mutable learning session. Serialize access in the host diff --git a/src/lib.rs b/src/lib.rs index 70075c2..ce06e2d 100644 --- a/src/lib.rs +++ b/src/lib.rs @@ -478,11 +478,21 @@ impl Predictor { /// Rebuilds what predicting uses after `learned` changes. Summed as /// floats: stored counts are bounded one by one, not in total. fn rebuild(&mut self) { - self.usable = Usable::default(); - let contexts: Vec> = self.learned.0.keys().cloned().collect(); - for context in &contexts { - self.rebuild_context(context); + let mut usable = Usable::default(); + for (context, words) in &self.learned.0 { + let kept: BTreeMap = words + .iter() + .filter(|(w, _)| self.usable_word(w)) + .map(|(w, c)| (w.clone(), *c)) + .collect(); + if !kept.is_empty() { + usable + .totals + .insert(context.clone(), kept.values().map(|c| *c as f64).sum()); + usable.counts.0.insert(context.clone(), kept); + } } + self.usable = usable; self.usable.learned = self .learned .0 @@ -492,7 +502,8 @@ impl Predictor { } /// Updates what prediction uses after `delta` was added to `learned`: /// its usable counts are added, and a word that has just become usable - /// gets all its learned counts. Counts are whole numbers, so adding them + /// gets all its learned counts. Counts are whole numbers, so while totals + /// stay below 2^53, which learning cannot pass, adding them /// in place gives the same totals as rebuilding. fn update(&mut self, delta: &Counts) { let empty = BTreeMap::new(); @@ -554,26 +565,6 @@ impl Predictor { .and_then(|w| w.get(word)) .is_some_and(|c| *c >= NEW_WORD_REPEATS) } - fn rebuild_context(&mut self, context: &[String]) { - let kept: BTreeMap = self - .learned - .0 - .get(context) - .into_iter() - .flatten() - .filter(|(w, _)| self.usable_word(w)) - .map(|(w, c)| (w.clone(), *c)) - .collect(); - if kept.is_empty() { - self.usable.totals.remove(context); - self.usable.counts.0.remove(context); - } else { - self.usable - .totals - .insert(context.to_vec(), kept.values().map(|c| *c as f64).sum()); - self.usable.counts.0.insert(context.to_vec(), kept); - } - } fn reweigh(&mut self) { self.usable.weight = PERSONAL_WEIGHT / (1. + self.usable.learned / PERSONAL_SATURATION); } @@ -778,26 +769,44 @@ mod tests { let dir = tempfile::tempdir().unwrap(); let base = dir.path().join("base.sqlite"); let personal = dir.path().join("personal.sqlite"); - build(&base, "hello water. i like tea. we like tea.", "fixture").unwrap(); + // "like" and "tea" are trusted: the baseline has them three times. + build( + &base, + "hello water. i like tea. we like tea. you like tea.", + "fixture", + ) + .unwrap(); + let imported = dir.path().join("imported.txt"); + fs::write(&imported, "zed likes tea. Aoife likes tea. Aoife said zed.").unwrap(); let mut p = Predictor::open(&base, Some(&personal)).unwrap(); + let check = |p: &mut Predictor, label: &str| { + let (counts, totals, weight) = ( + p.usable.counts.0.clone(), + p.usable.totals.clone(), + p.usable.weight, + ); + p.rebuild(); + assert_eq!(counts, p.usable.counts.0, "{label}"); + assert_eq!(totals, p.usable.totals, "{label}"); + assert_eq!(weight.to_bits(), p.usable.weight.to_bits(), "{label}"); + }; + assert!(p.import(&imported).unwrap()); + check(&mut p, "import"); + assert!(!p.import(&imported).unwrap()); + check(&mut p, "duplicate import"); for text in [ "Siobhan likes zed water.", "zed Siobhan likes tea.", "hello zed. Siobhan said hello.", "", "we like Siobhan and zed", + "you like tea.", ] { p.learn(text).unwrap(); - let (counts, totals, weight) = ( - p.usable.counts.0.clone(), - p.usable.totals.clone(), - p.usable.weight, - ); - p.rebuild(); - assert_eq!(counts, p.usable.counts.0, "{text}"); - assert_eq!(totals, p.usable.totals, "{text}"); - assert_eq!(weight.to_bits(), p.usable.weight.to_bits(), "{text}"); + check(&mut p, text); } + // Trusted words count from their first use. + assert!(p.usable.counts.0[&vec!["you".to_owned()]].contains_key("like")); assert!(p.usable.counts.0[&Vec::new()].contains_key("siobhan")); } diff --git a/tests/prediction.rs b/tests/prediction.rs index 7824c2c..e546449 100644 --- a/tests/prediction.rs +++ b/tests/prediction.rs @@ -697,9 +697,17 @@ fn prediction_cost_on_production_baseline() { p.learn(&text).unwrap(); p.learn(&text).unwrap(); let started = std::time::Instant::now(); - p.learn("I need to call the physio about my hoist.") + p.learn("I need to call the physio about my zzhoist.") .unwrap(); println!("{label}: one learn {:?}", started.elapsed()); + // A second use makes the new word usable, adding its counts in one pass. + let started = std::time::Instant::now(); + p.learn("I need to call the physio about my zzhoist.") + .unwrap(); + println!( + "{label}: learn making a word usable {:?}", + started.elapsed() + ); time(&p, label); } } From 45af5c098dd42755513d84ec6825a04f427dd7a7 Mon Sep 17 00:00:00 2001 From: Owen McGirr Date: Mon, 12 Oct 2026 16:38:54 +0100 Subject: [PATCH 6/8] Rewrap the learning paragraph Refs #29 Co-Authored-By: Claude Opus 5.5 --- docs/production.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/production.md b/docs/production.md index 5600402..3654cd0 100644 --- a/docs/production.md +++ b/docs/production.md @@ -52,7 +52,8 @@ keystroke incurs cold validation/loading. Loading the promoted model takes about 1.6 seconds on the development M2 Max. Learning writes only learned counts and validates the personal database: with a few thousand learned words a sentence takes tens of milliseconds, and about 200 ms at tens of thousands. It reloads -the personal database when another connection has changed it. Keep open/import/learn/reset/refresh off the UI thread. +the personal database when another connection has changed it. Keep +open/import/learn/reset/refresh off the UI thread. `Predictor` is an owned, mutable learning session. Serialize access in the host application; do not share an unsynchronized connection across threads. Multiple From e5223aa8af58d982278db110b70293546db1cdd7 Mon Sep 17 00:00:00 2001 From: Owen McGirr Date: Mon, 12 Oct 2026 16:44:56 +0100 Subject: [PATCH 7/8] Check trusted words on their first learned use Both tests that claimed to cover trusted words passed with the trusted path disabled, because the word had already been learned once. They now check a word's first learned use, and fail if trust is ignored. Refs #29 Co-Authored-By: Claude Opus 5.5 --- src/lib.rs | 6 ++++-- tests/prediction.rs | 10 +++++----- 2 files changed, 9 insertions(+), 7 deletions(-) diff --git a/src/lib.rs b/src/lib.rs index ce06e2d..0e831a6 100644 --- a/src/lib.rs +++ b/src/lib.rs @@ -804,9 +804,11 @@ mod tests { ] { p.learn(text).unwrap(); check(&mut p, text); + if text == "we like Siobhan and zed" { + // "like" is trusted, so its first learned use counts. + assert!(p.usable.counts.0[&vec!["we".to_owned()]].contains_key("like")); + } } - // Trusted words count from their first use. - assert!(p.usable.counts.0[&vec!["you".to_owned()]].contains_key("like")); assert!(p.usable.counts.0[&Vec::new()].contains_key("siobhan")); } diff --git a/tests/prediction.rs b/tests/prediction.rs index e546449..035ffaf 100644 --- a/tests/prediction.rs +++ b/tests/prediction.rs @@ -586,16 +586,16 @@ fn a_misspelling_learned_once_is_not_suggested_but_a_new_word_used_twice_is() { let (tmp, base) = fixture(); let personal = tmp.path().join("personal.sqlite"); let mut p = Predictor::open(&base, Some(&personal)).unwrap(); - p.learn("I drink teh water.").unwrap(); - assert!(!words(&p, "I drink ", "t").contains(&"teh".to_string())); - assert!(!words(&p, "", "te").contains(&"teh".to_string())); - // A word the baseline knows well counts at once; one it has rarely seen, - // which may itself be a misspelling, needs a second use. + // A word the baseline knows well counts from its first learned use; one + // it has rarely seen, which may itself be a misspelling, needs a second. assert_eq!(words(&p, "We walk ", "")[0], "home"); p.learn("We walk wine.").unwrap(); assert_eq!(words(&p, "We walk ", "")[0], "home"); p.learn("We walk drink.").unwrap(); assert_eq!(words(&p, "We walk ", "")[0], "drink"); + p.learn("I drink teh water.").unwrap(); + assert!(!words(&p, "I drink ", "t").contains(&"teh".to_string())); + assert!(!words(&p, "", "te").contains(&"teh".to_string())); p.learn("I drink teh tea.").unwrap(); assert!(words(&p, "I drink ", "t").contains(&"teh".to_string())); } From b1904a91104ef5e39a8dd98c6cdaa99dec550183 Mon Sep 17 00:00:00 2001 From: Owen McGirr Date: Mon, 12 Oct 2026 16:57:08 +0100 Subject: [PATCH 8/8] Name the measured code instead of this branch Refs #29 Co-Authored-By: Claude Opus 5.5 --- docs/personal-learning.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/personal-learning.md b/docs/personal-learning.md index 66a00fb..64decd0 100644 --- a/docs/personal-learning.md +++ b/docs/personal-learning.md @@ -57,7 +57,7 @@ is synthetic text imitating one person's typing; it is not user data. - A query is a hit when the word is in any of the six slots: 3 statistical and 3 generated. -Results from this branch's final commit, with v0.2.1 generation: +Results from #30's final code, with v0.2.1 generation: | Weighting | Top six | Personal words | Misspelling suggested | | --- | ---: | ---: | ---: | @@ -76,7 +76,7 @@ at 50×, 53.8% at 500×, 60.3% at 2,000× and 65.4% at 5,000×. General English must not suffer for someone who has learned text. 600 queries from the frozen general workload, with natural context, were run with that person's learning present. "Light" is the 40 original sentences learned once; -"heavy" is the same text learned ten times. On this branch, with v0.2.1 +"heavy" is the same text learned ten times. With #30's code and v0.2.1 generation: no learning 70.3%, light 70.7%, heavy 70.2%. These differences are within noise for 600 queries. The general-workload runs used a separate local harness, not this script.