Skip to content

fix: make auto-forward timing identical on every JavaScript host - #377

Merged
han4wluc merged 3 commits into
mainfrom
fix/portable-auto-forward-and-segmenter
Oct 18, 2026
Merged

han4wluc merged 3 commits into
mainfrom
fix/portable-auto-forward-and-segmenter

Conversation

@han4wluc

Copy link
Copy Markdown
Contributor

Problem

The auto-forward delay is base + readingUnits * 60 / multiplier. Two inputs of it come from the host, so the same line, text and settings get a different delay on different browsers, runtimes and runtime versions:

  1. Speed multiplier. getAutoForwardSpeedMultiplier returned 2 ** ((speed - 50) / 50). ECMA-262 leaves ** and Math.pow implementation-approximated, and engines, engine versions and C libraries disagree in the last bit. Only speeds 0, 50 and 100 are exact everywhere.
  2. Reading units. getAutoForwardReadingUnits used the host Intl.Segmenter whenever one existed; the bundled unicode-segmenter was only a fallback. Grapheme boundaries therefore follow the host's ICU / Unicode version.

The difference is observable: the playback scheduler converts delayMs to exact quanta (toExactDurationQuanta) and fires when remainingQuanta <= deltaQuanta, so a one-ulp different delay can fire on a different tick, and a different unit count changes the delay by 60 ms or more. It also means a non-JavaScript implementation of the engine cannot match "the" JavaScript result, because there is not one.

Evidence

Multiplier, compared with the correctly rounded value (the double nearest to 2^x), over all 101 integer speeds and 1,021,864 distinct fractional speeds (every hundredth, slider-style k * step values for 10 step sizes, a 10,000-case LCG set and 1,000,000 uniform random speeds). The correctly rounded reference was checked against an independent Python decimal computation at 80 digits on all 2,034,582 measured inputs, with 0 disagreements.

Host 2 ** x comes from Integer speeds off by 1 ulp Distinct speeds off by 1 ulp
Node 24.21 (V8 13.6), Linux x86-64 std::pow, glibc 2.43 0 / 101 832 / 1,021,864 (0.08%)
Node 24.21 with --no-use-std-math-pow V8's fdlibm port 9 / 101 (12, 25, 26, 27, 29, 62, 72, 76, 83) 100,699 (9.9%)
Node 22.12 (V8 12.4) V8's fdlibm port (no std::pow option) same 9 / 101 same as above
Bun 1.4.2 (JavaScriptCore), Linux glibc, except x = -0.5 1 / 101 (speed 25) 833

Examples: speed 25 is 0.7071067811865476 on Node 24 and 0.7071067811865475 on Node 22 and JavaScriptCore; speed 55.39 is 1.0775837498610183 on Node 24 where the nearest double is 1.0775837498610186. Every disagreement is exactly one ulp. Windows and macOS C libraries were not measured.

Reading units, host Intl.Segmenter against the bundled segmenter (unicode-segmenter 0.17.0, Unicode 17.0), on the 766 official Unicode 17.0 GraphemeBreakTest cases, a 44-text multilingual corpus (each raw and NFC) and 15,568,896 probes (every Unicode scalar value in 14 two- and three-character contexts covering combining marks, ZWJ, Indic virama, Hangul jamo, regional indicators, emoji modifiers, CR/LF and Thai SARA AM):

Host Unicode Official cases differing Corpus texts differing Probes differing
Node 24.21, ICU 78.3 17.0 0 0 0
Node 22.22, ICU 78.2 17.0 0 0 0
Bun 1.4.2, ICU 78.3 17.0 0 0 0
Node 22.12, ICU 76.1 16.0 10 2 2,976

On the Unicode 16.0 host, "ភាសាខ្មែរ" (Khmer) counts 5 units instead of 4, a Myanmar kinzi sequence 3 instead of 1, Gujarati and Balinese conjuncts one more, and U+2701 U+200D U+2701 one fewer (Unicode 17.0 changed GB9c conjunct data and Extended_Pictographic).

Fix

  • Segmentation: always split with the bundled splitGraphemes (already a dependency and already the fallback). NFC normalization and the unit weights are unchanged.
  • Multiplier: keep the clamp and (speed - 50) / 50 (IEEE-754 basic operations, identical everywhere) and return the correctly rounded 2^x. It is computed with exact BigInt fixed-point arithmetic: ln 2 from its series, e^(x ln 2) from the Taylor series with a proven error bound, and a retry at doubled precision whenever the error bracket straddles a rounding boundary. 2^x is irrational for every non-integer rational x, so it is never exactly halfway and the loop terminates; the first 128-bit pass decided all 2,034,546 measured inputs. About 90 lines, no new dependency, roughly 7 µs per call (called once per auto-forward timer).

A lookup table was considered and rejected: autoForwardSpeed accepts any number from 0 to 100 (schema type: number, sliders with a step other than 1, ${variables.*} bindings), so a table would need a fallback for fractional speeds. Because the correctly rounded value is unique, any correct implementation in another language reproduces these results bit for bit.

Compatibility

  • Integer speeds (the default slider step, every documented example): unchanged on Node 24 / V8 13.6 with glibc. They move by one ulp to the nearest double at speed 25 on JavaScriptCore and at 9 speeds on V8 builds that use the fdlibm port (Node 22).
  • Fractional speeds: 0.08% move by one ulp compared with Node 24 / glibc, about 10% compared with fdlibm-based hosts.
  • Resulting delays: across 80,008 cases (4 texts x speeds 0 to 100 in 0.01 steps x base 0 and 1000), at most 2 ulps and 7.3e-12 ms change (28 cases on Node 24, 4,405 on Node 22.12, 31 on Bun).
  • Reading units: unchanged on Unicode 17.0 hosts. Hosts with older ICU now count like Unicode 17.0.
  • No API change. The minified bundle grows by about 0.7 KB. BigInt is already used by playbackScheduler.js.
  • Upgrading unicode-segmenter to a newer Unicode version becomes an explicit behaviour change that the corpus test flags.
  • Not in this PR: the CJK weight test (\p{Script=Han} and friends), \s and normalize("NFC") still use the engine's built-in Unicode tables. Between the Unicode 16.0 and 17.0 hosts above, \s and single-code-point NFC were identical, but 4,321 code points newly assigned in Unicode 17.0 (U+16FF2..U+33479, mostly CJK Extension J) weigh 1 instead of 3 on the older host. A follow-up could bundle a Script range table.

Tests

spec/autoForwardTiming.test.js gains 40 tests:

  • the multiplier for all 101 integer speeds, and for 18 fractional speeds where glibc's pow misses the nearest double;
  • monotonic multipliers from speed 0 to 100 in 0.01 steps, ending at exactly 2;
  • delays derived from the exact multiplier at speeds 25, 55.39 and 93.5848910594359;
  • reading units for an 18-text corpus: Latin, combining and stacked marks, Han and kana, Hangul syllables and conjoining jamo, emoji ZWJ family, ZWJ with skin tone, flags, tag sequence, keycaps, Thai tone marks and SARA AM, Devanagari, Khmer and Myanmar conjuncts, U+2701 U+200D U+2701, CRLF and mixed dialogue;
  • a test that swaps Intl.Segmenter for a stub with outdated data (one code point per grapheme), re-imports the module and checks the stub is never constructed and the corpus counts do not change.

The old implementation fails the new tests where it depends on the host: 20 of 52 on Node 24.21, 17 on Node 22.12 (including the integer table and three Unicode 17 corpus rows) and 21 on Bun 1.4.2 (including speed 25). The new implementation passes 52 / 52 on all three. Full suite on Node 24.21: 2,366 passed before, 2,406 after (2,385 after the first commit alone). bun run lint, check:test-markers and format:changed pass.

A JSON vector set generated from this branch (25,538 multiplier rows, 754 delay rows, 25 reading-unit rows with grapheme boundaries) is attached for other implementations. It was cross-checked against the Python decimal reference and a literal Rust port of the same algorithm, with 0 mismatches.

Reading units came from the host Intl.Segmenter whenever one existed, so
the auto-forward delay of the same line depended on the browser's or
runtime's ICU data. Node 22.12 (ICU 76.1, Unicode 16.0) segments 10 of
the 766 official Unicode 17.0 GraphemeBreakTest cases and 2,976 of
15,568,896 code-point probes differently from Node 24.21 (ICU 78.3,
Unicode 17.0). Khmer, Myanmar, Gujarati and Balinese conjuncts and
U+2701 U+200D U+2701 change their unit counts; "ភាសាខ្មែរ" counts 5
units there and 4 on Unicode 17.0 hosts.

Always use the bundled unicode-segmenter (Unicode 17.0), which already
served hosts without Intl.Segmenter. On Unicode 17.0 hosts (Node 24.21,
Node 22.22 with ICU 78.2, Bun 1.4.2) it matches Intl.Segmenter on all
766 official cases, a 44-text multilingual corpus and every probe, so
results on those hosts do not change.
The multiplier was `2 ** ((speed - 50) / 50)`. ECMA-262 leaves `**` and
Math.pow implementation-approximated, and hosts disagree in the last
bit: V8 13.6 calls the C library's pow (glibc 2.43 on Linux), its own
fdlibm port (--no-use-std-math-pow) differs at 9 of the 101 integer
speeds, JavaScriptCore returns 0.7071067811865475 instead of
0.7071067811865476 at speed 25, and glibc itself misses the nearest
double for about 0.08% of fractional speeds (5.39, 44.62, 55.39, 61.09
and 76.42 among the hundredths). The playback scheduler compares
delays as exact quanta, so a one-ulp difference can move the tick on
which auto-forward fires.

Return the correctly rounded value instead. The clamp, subtraction and
division stay IEEE-754 basic operations, and 2 ** x is evaluated with
exact BigInt fixed-point arithmetic under a proven error bound, doubling
the precision whenever the bracket straddles a rounding boundary. The
result is unique, so any correctly rounded implementation, including
non-JavaScript ports, reproduces it bit for bit.

All 101 integer speeds keep the values V8 returns on Linux today; 832
of 1,021,864 distinct measured speeds move by one unit in the last
place to the nearest double. A call takes about 7 microseconds.
Graphemes already came from the bundled Unicode 17.0 segmenter, but their
weights still used the host: \p{Script=Han|Hiragana|Katakana|Hangul} and
\s follow the engine's regex Unicode tables, and normalize("NFC") the
host's normalization data. CJK Extension J (U+323B0..U+33479, new in
17.0) weighed 3 per character on Node 24 but 1 in Chromium 143, whose
regex data is older.

Weigh graphemes with bundled tables instead: the Han, Hiragana, Katakana
and Hangul ranges from Unicode 17.0 Scripts.txt (115,792 code points, an
exact match), and the ECMAScript WhiteSpace and LineTerminator code
points. Drop the NFC step: the bundled segmenter keeps bases and marks
together, and over every decomposable code point in conjunct, Hangul and
emoji contexts NFC only changed counts for U+2194 U+0338 next to a ZWJ
emoji sequence. Pin unicode-segmenter to 0.17.0 so the documented
Unicode version cannot move with a patch release.

Also cache the last speed multiplier, since timers re-read it on every
reconcile; correct the comment on V8's pow; spell out in the docs that
(speed - 50) / 50 is ordinary double arithmetic; restore Intl.Segmenter
correctly in tests on hosts without it; drop the test made redundant by
the host-segmenter stub; and escape invisible code points in the corpus.
@han4wluc
han4wluc merged commit ad0ed87 into main Oct 18, 2026
40 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant