Skip to content

perf: reduce sketch hashing overhead - #265

Merged
tisonkun merged 4 commits into
apache:mainfrom
tisonkun:codex/hash-performance
Sep 2, 2026
Merged

perf: reduce sketch hashing overhead#265
tisonkun merged 4 commits into
apache:mainfrom
tisonkun:codex/hash-performance

Conversation

@tisonkun

@tisonkun tisonkun commented Sep 2, 2026

Copy link
Copy Markdown
Member

Summary

  • improve hash-backed sketch update performance for integer and raw-byte inputs
  • preserve existing MurmurHash3 and XXHash64 outputs and streaming behavior
  • add end-to-end update benchmarks for every sketch using these hashers

Design Notes

The optimization keeps the public API and hash algorithms unchanged. It focuses on the existing hot path from a sketch update through Hash and the internal hasher, making that path easier for the compiler to specialize while keeping full-word processing separate from partial tails.

The benchmarks exercise public sketch APIs rather than isolated hash helpers. Integer inputs benefit most from exposing the complete fixed-size hashing path to the optimizer. Raw-byte inputs additionally benefit from exposing the single contiguous write and using fixed-width reads for complete words. Count-Min performs four seeded MurmurHash3 passes per update in this benchmark, while Bloom performs two XXHash64 passes, so reductions in per-hash overhead accumulate in those sketches.

Benchmarks

Local Divan benchmark medians for 10,000 updates:

Sketch Input Before After Change
Bloom u64 383.2 us 303.5 us 20.8% faster
Bloom 32-byte raw input 513.5 us 398.6 us 22.4% faster
Count-Min u64 268.5 us 70.53 us 73.7% faster
Count-Min 32-byte raw input 281.9 us 242.0 us 14.2% faster
CPC u64 128.8 us 89.42 us 30.6% faster
CPC 32-byte raw input 144.4 us 139.8 us 3.2% faster
Frequent Items u64 338.8 us 129.4 us 61.8% faster
Frequent Items 32-byte raw input 397.0 us 388.1 us 2.2% faster
HLL u64 148.2 us 96.06 us 35.2% faster
HLL 32-byte raw input 174.4 us 143.9 us 17.5% faster
Theta u64 120.1 us 49.57 us 58.7% faster
Theta 32-byte raw input 120.0 us 108.9 us 9.3% faster
Tuple u64 125.3 us 56.78 us 54.7% faster
Tuple 32-byte raw input 124.5 us 116.8 us 6.2% faster

The baseline uses the same benchmark code with datasketches/src/hash restored from main; the optimized run uses this PR. Results are machine-dependent and document local before-and-after evidence rather than a portable performance guarantee.

Validation

  • cargo x check
  • cargo x test
  • cargo x lint
  • cargo bench --package benchmarks --bench benchmarks -- bloom::update countmin::update cpc::update frequencies::update hll::update theta::update tuple::update --min-time 1 --sample-count 20

@tisonkun
tisonkun merged commit 3d8b664 into apache:main Sep 2, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant