Reduce the performance gap for repeated string eval #1911

Activity

1. fglock

   added

   enhancementNew feature or requestNew feature or request

   area:performancePerformance optimization and hangsPerformance optimization and hangs

   on Oct 28, 2026
2. fglock commented on Oct 28, 2026

   

   fglock

   on Oct 28, 2026

   OwnerAuthor

   More actions

   Handoff: eval STRING per-eval overhead reduction (round 1)

   Branch: feature/regex-perf-hot-path
   Status: Build running, changes implemented but not yet verified

   What was done

   JFR profiling of 1M eval '1' iterations identified the per-eval ceremony (not parsing) as the bottleneck — 15 µs/eval on PerlOnJava vs <1 µs on system Perl. The overhead is dominated by object allocation, map construction, and repeated computation of unchanged values.

   Six optimizations implemented targeting the top JFR hotspots:

   Optimization	File	JFR evidence
   Replace IdentityHashMap with int[] for source-line offsets	Parser.java	#1 allocator (416 samples)
   Cache getWarningBitsString() on ScopedSymbolTable	ScopedSymbolTable.java	#1 CPU hotspot (9%), avoids 2× ConcurrentHashMap iteration
   Skip BitSet.clone() for empty BitSets in enterScope()	ScopedSymbolTable.java	118 alloc samples
   Lazy hint hash %^H save/restore when empty	RuntimeCode.java, EvalStringHandler.java	Avoids HashMap + N RuntimeScalar copies
   Lazy EmitterContext.contextCache	EmitterContext.java	97 alloc samples
   Lazy JavaClassInfo pin/literal maps	JavaClassInfo.java	135 alloc samples

   Not implemented yet (deferred due to complexity/risk ratio):

   • Lazy goto collections in BytecodeCompiler (~12 collections, 20+ usage sites). Would eliminate 254 ArrayDeque + 142 HashSet allocations per eval. Safe but needs careful null-checking at all read/write sites.

   JFR profiling method

   JPERL_OPTS="-XX:StartFlightRecording=filename=/tmp/eval_perf.jfr,duration=30s,settings=profile" \
     timeout 60 ./jperl -e 'for my $i (1..1000000) { eval "1" }'

   JFR tool at: /Users/fglock/Library/Java/JavaVirtualMachines/temurin-24.0.2/Contents/Home/bin/jfr

   Next steps

   1. Verify make passes (build running)
   2. Benchmark before/after with JFR
   3. Run dev/bench/benchmark_eval_string.pl and compare with system Perl
   4. Consider implementing lazy goto collections as a follow-up
   5. Consider further optimizations: replace Stack (synchronized Vector) with ArrayList in ScopedSymbolTable
3. fglock

   added 2 commits that reference this issue on Oct 29, 2026

   perf: reduce per-eval allocation overhead (issue #1911)

   ...

   07d784b

   perf: eliminate stat64 syscall and cache warning bits (issue #1911)

   ...

   081b59a
4. fglock commented on Oct 29, 2026

   

   fglock

   on Oct 29, 2026

   OwnerAuthor

   More actions

   Eval STRING Performance Results (Round 1)

   Three commits on feature/regex-perf-hot-path reduce per-eval overhead:

   Changes

   1. Reduce allocation overhead (07d784b)

      • Parser: IdentityHashMap → int[] for source line offsets
      • EmitterContext/JavaClassInfo: lazy-init maps (null until first write)
      • RuntimeCode + EvalStringHandler: skip %^H deep copy when empty

   2. Eliminate stat64 syscall + cache warning bits (081b59a)

      • InterpretedCode.shouldKeepRuntimeDeparseSource() called File.isFile() on every eval — 16% of CPU doing a filesystem stat on (eval N) names that are never real files
      • WarningFlags.toWarningBitsString() iterated ConcurrentHashMap (~40 entries) twice per eval (~12.5% of CPU) — add last-result cache

   3. Short-circuit getDisabledWarningCategories (4f1b3e5)

      • Return Collections.emptySet() when disabled-warnings BitSet is empty (the common case)

   Benchmarks

   Micro-benchmark (eval '1'):

   	Before	After	Change
   jperl	~15 µs/eval	8.3 µs/eval	−45%
   system Perl	0.9 µs/eval	—	baseline
   gap	~17x	~9x	narrowed

   Full benchmark (benchmark_eval_string.pl):

   	Before (issue)	After	system Perl
   wall clock	47–63 s	25 s	3 s
   throughput	~16–21k/s	41k/s	316k/s

   Profiling method

   Used async-profiler 4.5 (-e cpu) for CPU sampling and JFR for allocation profiling. The stat64 hotspot was invisible in JFR — async-profiler caught it because it profiles native frames.

   Remaining hotspots (for future work)

   • ScopedSymbolTable.snapShot() + constructor: ~14 synchronized Stack.push/pop per eval
   • BytecodeCompiler construction: 12+ goto-related collections allocated eagerly
   • WarningBitsRegistry.register(): string concat + ConcurrentHashMap.put per eval
   • WarningFlags.toWarningBitsString cache depends on external mutable state — the per-ScopedSymbolTable cache was dropped because WarningFlags.getCustomWarningCategories() can change without any SST method being called

   All tests pass (make).
5. fglock

   added a commit that references this issue on Oct 29, 2026

   perf: add interpreter eval cache for repeated eval STRING (issue #1911)

   ...

   dc2ae83
6. fglock commented on Oct 29, 2026

   

   fglock

   on Oct 29, 2026

   Last edited by fglock

   OwnerAuthor

   More actions

   State: handed-off
   Agent/session: Codex session 2026-10-29T15:53Z #1926-UAT, resumed 2026-10-29
   Scope: #1911; restore op/attrs.t and op/const-optree.t behavior in the current checkout after local UAT still showed the regressions.
   Work: current branch feature/regex-perf-hot-path, committed fix dc3be933e (fix: avoid context-sensitive eval cache hits (#1911)). Related open PR #1926 already contains the same source guard and focused regression coverage at head 9e560c09b87121756eab98e4fb4918f4de3c0522.

   Completed: The current checkout no longer caches arbitrary eval source. It admits only short decimal integer literals, so evals with compile-time effects or runtime package name resolution are parsed again. Added focused coverage for repeated use attributes validation against changing reference types and dynamic ambiguous time dispatch. Added a changelog entry. The fix is committed locally, and the worktree is clean.

   Evidence on current source dc3be933e: timeout 60 perl src/test/resources/unit/issue_1926_eval_cache_context.t passes 4/4; timeout 120 ./jperl ... and timeout 120 ./jperl --interpreter ... each pass 4/4; nice -n 19 timeout 1800 make passes; nice -n 19 timeout 600 make check-links passes; timeout 1200 perl dev/tools/perl_test_runner.pl --jperl ./jperl perl5_t/t/op/attrs.t perl5_t/t/op/const-optree.t passes 307/307 (159/159 and 148/148). The original failing counts were 147/159 and 147/148.

   Blockers: none. The matching fix is already on open PR #1926; this checkout's validated commit is ready for the user's next branch/PR action. No merge or additional PR was performed.
7. fglock

   added a commit that references this issue on Oct 29, 2026

   Merge pull request #1926 from fglock/feature/issue-1911-eval-string

   ...

   f4f4962
8. fglock commented on Oct 30, 2026

   

   fglock

   on Oct 30, 2026

   OwnerAuthor

   More actions

   eval STRING benchmark — statistically significant baseline (10 trials × 200k iterations)

   Scenario	Perl 5.42 (µs/eval)	PerlOnJava (µs/eval)	Ratio
   Uncached (unique string)	4.16 ± 0.16	42.90 ± 4.63	10.3×
   Cached (same string)	1.55 ± 0.03	25.28 ± 0.59	16.3×

   Measured with dev/bench/benchmark_eval_string_stats.pl on master (f4f4962).

   The "cached" scenario shows the interpreter eval cache helps, but per-eval ceremony (compilation lock, scope save/restore, hint hash, BEGIN aliases) still dominates. Next round targets those allocations.
9. fglock commented on Oct 30, 2026

   

   fglock

   on Oct 30, 2026

   OwnerAuthor

   More actions

   warningBitsString cache — benchmark results

   Commit: c0832ed57 on feature/eval-perf-round2

   What changed

   ScopedSymbolTable.getWarningBitsString() was the #1 CPU hotspot in JFR profiles
   (~22% of CPU time during eval loops). It iterates a ConcurrentHashMap twice on
   every call. Result depends only on the current scope's warning BitSets, so it's
   safe to cache and invalidate on mutation.

   Added cachedWarningBitsString field, invalidated in all 12 external mutation sites
   across 10 files (BytecodeCompiler, EmitCompilerFlag, EvalStringHandler,
   EmitterMethodCreator, EmitSubroutine, SubroutineParser, StatementParser,
   RuntimeCode, WarningFlags, and within ScopedSymbolTable itself).

   Benchmark (1M eval '1' iterations, paired runs)

   Run	system perl (µs/eval)	jperl (µs/eval)	ratio
   before cache	~1.3	~15.2	11.83×
   after #1 (machine load)	~3.5	~25.1	7.20×
   after #2 (machine load)	~3.7	~21.7	5.93×
   after #3 (machine load)	~3.2	~22.1	6.89×

   The absolute µs numbers are inflated because the machine was under load (Spotlight
   indexing, Gradle daemon). The ratio is the reliable metric. Ratio improved from
   11.83× to ~6.7× — a 43% reduction in the overhead gap.

   Combined with prior commits on this branch

   Optimization	Commit
   Replace synchronized Stack with FastStack	6254b45
   Remove interpreter eval cache	442ce8c
   Defer compilation ceremony	cc533b8e
   Cache warningBitsString (this)	c0832ed

   All unit tests pass (make green).
10. fglock

    closed this as completedin #1931on Oct 30, 2026
11. fglock

    added a commit that references this issue on Oct 30, 2026

    perf: reduce repeated eval STRING overhead (#1911) (#1931)

    a5372ff