Repository navigation
Conversation
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #72 (
perf/hot-paths); the diff here is one reader change plus tests.Parsing an array of one million decimals took 246 ms on
perf/hot-pathsagainst 146 ms for jq 1.7. A stage-by-stage harness over the same file showed where the time went: lexing plus list building alone costs about 117 ms and 7 words per number, decimal construction (Decimal.of_lexeme_opt) only about 24 ms, and the remaining hundred milliseconds is allocation. The real path allocates 26 words per number, 3.6x the stub, mostly fromLexing.lexemecopying every literal into a fresh string kept asreprfor round-trip printing, plus the record and itsSomebox. A larger minor heap did not help (s=4Mwas 17 % slower), andZ.of_intallocates nothing for values in range, so a small-coefficient variant would gain nothing.Change.
Read.mllgainstry_fast_decimal: for a literal of the shape-?digits.digitswith no exponent, at most 18 significant digits and no trailing zero in the fraction, it accumulates the digits straight fromlexbuf.lex_bufferand builds{ coeff; scale; repr = None }with no string allocation. The grammar'spositive_intalready forbids leading zeros, so for that shape canonical printing is byte-identical to the source text; the trailing-zero case (1.50) is the only one where it is not, and it falls back. Exponents, longer mantissas and trailing zeros take the unchanged slow path withreprpreserved. No type or consumer changes.Measured,
hyperfine -N --warmup 3 --min-runs 15, release builds, Linux x86-64,-c lengthon the 1M-float array (0f5ca93ismain,ecae5a8is the base of this PR):0f5ca93(main)ecae5a8(perf/hot-paths)On a 200k-record array with three decimal fields each, 264 to 213 ms (jq 183). Integer parsing is untouched (1M ints: within noise). Arithmetic-heavy
map(. + 10) | addgains about 6 %, since compute dominates there.Parity.
cmpof-coutput against a0f5ca93build for.,map(. + 10) | length,add,maxandsort | .[0:3]on the 1M-float array, 1M ints, the 21 MB object array,small.json,big.jsonandhuge.json: 30 of 30 identical. Literal round trips identical for1.0,1e5,-0,-0.0,1.50,1E-7,1.5e-7,0.005,100.00,-0.5and a 21-digit mantissa. Structural-equality cases ([1.5, 1.50] | unique,[1.5, 0.5 + 1.0] | unique,group_by(.)) unchanged. Exact decimal arithmetic unchanged.Checks at this head:
dune build @runtest @fmtexit 0 (847 runtime tests, 104 reader tests incl. 4 new for both sides of the fast path, Cram), release@allexit 0,git diff --checkclean.Remaining gap. The rest of the distance to jq and jaq is the list-of-boxed-values representation and cons-then-reverse array building in the reader, which affects every type, not decimals; that is a representation redesign and out of scope here.
Risk: two-way door. A merge revert restores the previous reader path; no data, API or output bytes change.