diff --git a/README.md b/README.md index b45f930..8475c25 100644 --- a/README.md +++ b/README.md @@ -99,12 +99,15 @@ limit: | Claim | What hvec measures | What that does not show | |---|---|---| | **Less memory at comparable quality** | Bytes per vector, and nDCG@k and document recall@k against the labels, next to the f32 ceiling | Anything about speed | -| **Faster search** | Scan time of the compressed-domain score over the whole collection | Not yet claimable: hvec scores every codec asymmetrically against a float query in a brute-force scan, so binary is no faster than int8 here, and nothing in the harness is memory-bandwidth bound. A speed claim needs a symmetric (Hamming, lookup-table) path and a setting where bytes read is the bottleneck | +| **Faster search** | Scan time per query over the whole collection, in place in a contiguous buffer (`bench scan`), against the machine's memory bandwidth | Only the symmetric `binary-hamming` scan is faster: every asymmetric codec costs about what f32 costs per vector, because its scoring, not its bytes, sets the time. Measured on one machine, brute force, with the hybrid's re-score cost computed rather than timed end to end | | **Useful operations directly on the compressed vectors** | Every codec's `score` runs without decoding, and is tested to equal the metric on the decoded vector | Whether the operation is cheap; see the speed row | Where the published numbers stand on the first claim: `int8` is 4× smaller and indistinguishable from f32 against the labels; `pq48` is 32× smaller at two points of nDCG@10; `binary` is 32× -smaller at 4.4 points. On the second claim hvec makes no claim yet. The useful advantage a new +smaller at 4.4 points. On the second claim: a `binary-hamming` scan plus an f32 re-score of 369 +candidates reaches 95% recall@10 on FiQA about 19× faster than an exact f32 scan on one thread, +and is the only compressed path that is faster at all +([zero-copy scan](experiments/2026-11-zero-copy-scan/)). The useful advantage a new codec has to demonstrate is over *those* numbers, on the same corpus, with the byte count stated. ### 2. Does the loss reach the answer? @@ -339,6 +342,11 @@ parameters alongside the collection, and reuses them for every later ingest, so call a representative sample of the corpus rather than a single file. `mrl` keeps the first `k` dimensions (what Matryoshka-trained models are built for) and composes with any other codec: `mrl256+int8`, `mrl128+pq16`. +`binary-hamming` stores the same bits as `binary` but binarises the query too and scores by XOR +and popcount: coarser, and about 20× faster to scan. `mixed` is trained: it spends a fixed +`B` bytes per vector with bits allocated per +dimension by variance. `mixed-pca` rotates into the corpus's principal components first, and +`-w` weights each component by how much energy queries put into it (see below). With a question set that carries relevance judgments, `bench recall --questions` also reports nDCG@k and document recall@k against the labels, for every codec and for the f32 ceiling: @@ -373,8 +381,67 @@ recovers only four or five points of recall: one bit per dimension cannot separa whose cosines differ by less than its resolution, whatever the origin is. Against the human relevance judgments the picture is gentler: binary loses 4.4 points of nDCG@10 (0.698 to 0.654), not 39, because the neighbours it reorders are near-ties that are rarely the judged abstract. -The f32 scan is slow only because the baseline decodes each vector; the other codecs score the -bytes directly. The experiments are written up under [`experiments/`](experiments/). +The scan times in that table predate the in-place f32 kernel, when f32 decoded every vector into +a new allocation. `bench scan` is now the place to time scans: it scores every codec in place in +a contiguous buffer and reports GB/s against the machine's memory bandwidth +([zero-copy scan](experiments/2026-11-zero-copy-scan/)). The experiments are written up under +[`experiments/`](experiments/). + +#### Hybrid and adaptive quantization + +A small code scanned over every vector loses neighbours, and how many it loses is predictable: +it is set by the code's score noise divided by how tightly the corpus packs a query's +neighbours (the [neighbour-spacing experiment](experiments/2026-11-neighbour-spacing/)). There +are two ways to win the loss back. + +**Hybrid.** Keep the small code in memory for the scan, shortlist its best N candidates, and +re-score only those with int8 or f32 vectors fetched from disk: + +

+ Animated schematic of two-stage retrieval: every vector is scored from a 48-byte code; the code's top 10 keeps 6 of the 10 true neighbours; the code's top 30 are re-scored with exact f32 vectors, lifting 9 of the 10 true neighbours into the top 10, while one ranked 31st by the code falls outside the shortlist and is missed. +

+ +```sh +hvec bench rescore --collection scifact --questions questions/scifact.jsonl \ + --codecs binary,pq48 --rescore int8,f32 --depths 20,50,100 +``` + +Across ten collections (five corpora, two embedding models), re-scoring 50 f32 candidates +behind a 48-byte scan recovers 0.95 (`binary`) to 0.99 (`pq48`) of the exact top 10 and +essentially all of the nDCG loss. That costs 3–4% of the bytes a full f32 scan reads. The depth +needed follows the same noise-to-spacing ratio (Spearman ρ ≈ 0.93), so one f32 pass over a +corpus tells you how deep to go: +[two-stage re-scoring](experiments/2026-11-two-stage-rescoring/). + +**Adaptive depth.** A fixed N is too deep for some queries and too shallow for others. The +`--margins` policy re-scores every candidate whose code score is within λσ of the code's own +10th score, where σ is the code's noise, calibrated once per collection on self-queries: + +

+ Animated schematic of adaptive re-scoring depth: two queries share the same noise-based margin below their 10th code score; a crowded query has 59 candidates inside the margin and a well-separated query 18, while a fixed shortlist of 30 would under-fetch for the first and over-fetch for the second. +

+ +Measured exactly at the same mean depth, the margin policy adds only 0.3 to 0.6 points of +recall@10 over a fixed depth, but up to 4 points for the worst decile of queries. A hindsight +oracle shows about 6 points of headroom, almost none of which is visible from the scan's own +scores ([routing experiment](experiments/2026-11-routing-oracle/)). + +**Adaptive bits.** Spend the scan code's bytes unevenly, where they buy the most. This is the +idea behind mixed-precision quantization of mixture-of-experts models, where blocks and experts +that matter more get more bits ([QuantMoE-Bench](https://github.com/UNITES-Lab/moe-quantization)). +For embeddings, "matters more" means a principal component with more variance, or with more of +the query's energy: + +

+ Animated bar chart of how a 48-byte budget is spread over 384 principal components of SciFact's bge-small embeddings: uniform binary gives every component 1 bit (recall@10 0.611); variance-based allocation concentrates bits on the leading components and gives 97 none (0.694); query-weighted allocation gives 158 none and returns a few bits to the last components, which carry 28% of the corpus mean (0.703). +

+ +On SciFact at 48 bytes with no re-scoring, `mixed48-pca-w` keeps 0.703 of the top 10, against +0.611 for `binary` and 0.711 for `pq48`. The full comparison, every scan codec against every +re-score policy, is the [hybrid × adaptive matrix](experiments/2026-11-hybrid-adaptive-matrix/). +The animations are generated by +[`docs/assets/animations/make_animations.py`](docs/assets/animations/make_animations.py); the +bit-allocation chart uses SciFact's real PCA spectrum and allocations. ### 7. Run the matrix and report diff --git a/ROADMAP.md b/ROADMAP.md index 35749fc..ef61a14 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -107,7 +107,9 @@ rotation is a `d × d` matrix fitted on the corpus. `rot+pq48`, `mrl256+rot+binary`. It is a trained codec in the existing sense: `fit` learns the rotation on the first ingest, `params` stores the matrix (or the seed) with the collection, and `score` rotates the f32 query once per search before the asymmetric scan, so the stored - side still scores on the bytes. + side still scores on the bytes. The hook for that now exists (`Codec::prepare_query` and + `score_prepared`, added in phase 8), and `mixed-pca` already carries a PCA fit that + `rot:itq` can start from. - [ ] Two fits behind the same name. `rot` with a seed is a random orthogonal rotation (RaBitQ-style, a Haar-random matrix from the QR of a Gaussian), which needs no corpus pass and gives a baseline for how much the learned one buys. `rot:itq` is Iterative Quantization: PCA @@ -127,6 +129,78 @@ rotation is a `d × d` matrix fitted on the corpus. Exit: Experiment 4 published, and a stated answer to whether rotation makes binary usable on this embedding model. +## Phase 8 · Spacing, hybrid and adaptive quantization, routing, zero-copy (done; follow-ups open) + +Every codec number before this phase came from one corpus, so none of them said what a codec +would cost elsewhere. This phase finds the quantity that sets the cost, then uses it twice: to +decide how deep a hybrid re-score must go, and to decide where an adaptive code spends its bits. + +- [x] Neighbour spacing in `bench recall`: `neighbour_band` (exact score gap from rank k to 2k), + `boundary_gap` (k to k + 1) and `rank_noise` (the codec's score-error spread over the exact + top 2k, per-query bias removed). +- [x] **Neighbour-spacing experiment** (`experiments/2026-11-neighbour-spacing`): SciFact, + NFCorpus, ArguAna, FiQA and CoSQA (code, via CoIR) × bge-small and MiniLM × nine codecs. + Noise ÷ band ranks recall@10 loss with ρ = 0.98 over 90 points (noise alone 0.64) and predicts + a held-out corpus within 0.024. Content type matters only through spacing, and the embedding + model moves spacing as much as the corpus does. Published with a site page and a post. +- [x] `hvec bench rescore`: scan with one codec, re-score candidates with another (int8 or f32), + and report recall, worst-decile recall, nDCG, depth and bytes read per query. It also reports + the exact depth an f32 re-score needs for 90/95/99/100% recall. +- [x] **Two-stage re-scoring experiment** (`experiments/2026-11-two-stage-rescoring`): the + depth needed tracks the same ratio (ρ ≈ 0.93; depth ≈ 4.8·k·r^1.29 for 95% recall). At 48 + bytes, 50 f32 candidates recover 0.95 (`binary`) to 0.99 (`pq48`) of the top 10 and nearly all + of the nDCG loss, for 3–4% of an f32 scan's bytes. An int8 second stage stops at int8's own + 0.990. +- [x] Adaptive depth: a `--margins` policy that re-scores every candidate within λσ of the + scan's own 10th score, with σ calibrated on self-queries only. +- [x] Adaptive bits: `mixed[-pca][-w]`, a fixed byte budget with bits allocated per + dimension by greedy reverse water-filling and a Gaussian-optimal quantizer step. `-pca` + rotates into principal components first; `-w` weights each component by query energy, which + targets score error rather than reconstruction error. On SciFact at 48 bytes with no + re-score: `binary` 0.611, `mixed48` 0.653, `mixed48-pca` 0.694, `mixed48-pca-w` 0.703, `pq48` + 0.711. +- [x] `Codec::prepare_query` / `score_prepared`, so transformed-space codecs rotate the query + once per search; used by the store's search, `bench recall` and `bench rescore`. +- [x] Animated README figures for hybrid re-scoring, adaptive depth and adaptive bits + (`docs/assets/animations/`, regenerated by `make_animations.py`). +- [x] **Hybrid × adaptive matrix** (`experiments/2026-11-hybrid-adaptive-matrix`): 11 scan codecs + × none/fixed/margin × int8/f32 over ten collections. `mixed48-pca-w` beats `pq48` at equal + bytes on average (0.768 vs 0.747 recall@10; the gain is all on MiniLM, a tie on bge-small), and + leads at 24 and 96 bytes too. Adaptive depth gains 0.3–0.6 points of recall at equal mean + depth, and up to 4 points for the worst decile. (The first version said 1.5 points; that came + from interpolating between sparse fixed depths, and is corrected in the write-up.) +- [x] Per-query traces (`bench rescore --trace`) and the **routing experiment** + (`experiments/2026-11-routing-oracle`). A hindsight oracle has 5.5–7 points of headroom over + fixed depth. The band visible in the scan's own scores tracks the true band only at Spearman + 0.52–0.70, and no router built on it captures more than 6%. Per-collection routing picks the + right codec size in 10/10 collections, but its predicted depth misses a 95% target by up to + 9 points. +- [x] **Zero-copy** (`experiments/2026-11-zero-copy-scan`). Borrowed codes in the `Codec` trait, + `CodeBuffer` (a contiguous, cached per-collection buffer scanned in place, optionally over + threads), in-place f32 and single-pass cosine kernels, and `binary-hamming` (popcount). The + cached buffer is 5.1× faster than the original row scan. Every asymmetric codec costs about + what f32 costs per vector. `binary-hamming` is 20× cheaper, and with an f32 re-score it reaches + 95% recall@10 on FiQA about 19× faster than exact f32: the first speed claim hvec makes. +- [ ] Choose the codec automatically at ingest from noise ÷ band, then measure the re-score + depth for a target recall on a sample of self-queries. The routing experiment showed that the + curve chooses the codec well but predicts the depth badly. +- [ ] Layered (residual) codes: one stored code read to different depths per candidate, so + hybrid retrieval needs no second full-precision copy on disk. +- [ ] A faster `mixed` scan (byte-aligned groups, or a lookup table per group as PQ uses): + today it is the best 48-byte code for recall and the slowest to scan, 2.2× f32 per vector. +- [ ] Time the hybrid end to end, with the f32 candidates fetched from disk, not memory. +- [ ] SIMD kernels for f32 and int8, so the asymmetric codecs are compared at their best. +- [ ] Decide whether to allow one reviewed `unsafe` call for memory-mapped code files + (`unsafe_code` is forbidden workspace-wide today). A flat file already loads 110 MB in 9 ms, so + the case is memory footprint and start-up on much larger collections. +- [ ] Per-vector adaptive bits, the analogue of MoE PTQ's outlier scorer: more bits for vectors + in crowded neighbourhoods. Needs a variable-size encoding or two tiers. +- [ ] Per-cluster PCA (Sub-MoE's grouped subspaces), if `mixed-pca` gets within reach of + `pq` at equal bytes. +- [ ] Spacing follow-ups the write-up flagged: a deduplicated code corpus (CoSQA is 70% + duplicates), FiQA subsampled to 8,000 chunks to separate corpus size from spacing, a third + embedding model, and a tail statistic for the aggressive codecs the curve misses. + ## Ideas from MoE compression Weight compression for Mixture-of-Experts models is working on a problem shaped like this one: @@ -160,13 +234,11 @@ Three lines of work, and what each would mean as an hvec codec or experiment: - Run Experiments 1 and 3 and the judge re-scoring as soon as two chat models are reachable; the scripts need only `ANTHROPIC_API_KEY` (or an Ollama server) and a built `hvec`. -- Codec follow-ups the write-ups flagged: a per-vector scale for `binary-centred`, and a - rescoring stage (fetch 5-10× candidates by codec, re-rank with int8 or f32). -- A second corpus with graded, denser judgments than SciFact's. -- Done 2026-11-04: neighbour spacing across five corpora (`experiments/2026-11-neighbour-spacing`). - Codec noise ÷ neighbour band predicts recall@10 loss (ρ = 0.98). Follow-ups: rescoring depth - as a function of that ratio, a deduplicated code corpus, and FiQA subsampled to separate - corpus size from spacing. +- Phase 8 follow-ups, starting with choosing the codec at ingest and layered codes. +- Codec follow-ups the write-ups flagged: a per-vector scale for `binary-centred`. The + re-scoring stage is done (phase 8). +- Graded relevance: NFCorpus ships graded judgments, but hvec's nDCG treats relevance as + binary. Use the grades, then re-score the five-corpus results. ## Always diff --git a/crates/hvec-bench/src/rescore.rs b/crates/hvec-bench/src/rescore.rs index eb1a594..728349b 100644 --- a/crates/hvec-bench/src/rescore.rs +++ b/crates/hvec-bench/src/rescore.rs @@ -77,6 +77,37 @@ pub struct Plan<'a> { pub noise: f64, /// Cap on any policy's per-query depth. pub max_depth: usize, + /// When non-empty, record a [`QueryTrace`] per query, with the margin + /// policy's depth at each of these margins. + pub trace_margins: &'a [f64], +} + +/// Everything about one query that a routing policy could use or be judged +/// by, so policies can be replayed offline without re-running the scan. +/// +/// With an exact second stage, the query's recall at depth `N` is the number +/// of `ranks` at most `N`, divided by k: any fixed, adaptive or oracle depth +/// can be scored from this alone. +#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)] +pub struct QueryTrace { + /// First-stage rank (1-based) of each exact top-k neighbour, in exact order. + pub ranks: Vec, + /// Exact score gap from rank k to rank 2k: the true neighbour band. + pub band: f64, + /// The same gap measured on the first stage's own scores: the band a + /// router can see at query time without any exact scores. + pub band_hat: f64, + /// The margin policy's depth at each of the plan's `trace_margins`. + pub margin_depths: Vec, +} + +/// The result of [`evaluate`]. +#[derive(Debug, Clone)] +pub struct Evaluation { + pub profile: FirstStageProfile, + pub reports: Vec, + /// One per query when the plan asked for traces, otherwise empty. + pub traces: Vec, } /// One (first stage, second stage, policy) cell. @@ -145,7 +176,7 @@ pub fn evaluate( k: usize, plan: &Plan<'_>, sources: Option<&[String]>, -) -> Result<(FirstStageProfile, Vec), CodecError> { +) -> Result { assert_eq!(ids.len(), vectors.len(), "ids and vectors must align"); if let Some(s) = sources { assert_eq!(s.len(), ids.len(), "ids and sources must align"); @@ -197,6 +228,7 @@ pub fn evaluate( let mut bands: Vec = Vec::with_capacity(queries.len()); let mut noise_sum = 0f64; let mut first_recall_sum = 0f64; + let mut traces: Vec = Vec::new(); for q in queries { // Scores indexed like `ids`; the excluded id scores worst everywhere. @@ -215,9 +247,16 @@ pub fn evaluate( let truth: Vec = exact_order.iter().take(k).map(|&i| ids[i]).collect(); // Spacing and noise, exactly as `recall::evaluate` measures them. + let mut band = 0.0; + let mut band_hat = 0.0; if exact_order.len() >= 2 * k { let s = |r: usize| f64::from(exact[exact_order[r - 1]].1); - bands.push((s(k) - s(2 * k)).abs()); + band = (s(k) - s(2 * k)).abs(); + bands.push(band); + } + if n_live >= 2 * k { + let h = |r: usize| f64::from(approx[approx_order[r - 1]].1); + band_hat = (h(k) - h(2 * k)).abs(); } noise_sum += centred_noise(&exact, &approx, &exact_order[..exact_order.len().min(2 * k)]); @@ -229,6 +268,7 @@ pub fn evaluate( let ranks: Vec = exact_order.iter().take(k).map(|&i| rank_of[i]).collect(); first_recall_sum += ranks.iter().filter(|&&r| r <= k).count() as f64 / k as f64; full_depths.push(ranks.iter().copied().max().unwrap_or(0) as f64); + let trace_ranks = (!plan.trace_margins.is_empty()).then(|| ranks.clone()); pooled_ranks.extend(ranks); // The adaptive policy's depth: candidates within the margin of the @@ -243,6 +283,14 @@ pub fn evaluate( }; n.max(k).min(cap.min(n_live)) }; + if let Some(ranks) = trace_ranks { + traces.push(QueryTrace { + ranks, + band, + band_hat, + margin_depths: plan.trace_margins.iter().map(|&m| within(m)).collect(), + }); + } let is_labelled = sources.is_some() && !q.relevant.is_empty(); if is_labelled { @@ -357,7 +405,11 @@ pub fn evaluate( } }) .collect(); - Ok((profile, reports)) + Ok(Evaluation { + profile, + reports, + traces, + }) } #[cfg(test)] @@ -407,8 +459,9 @@ mod tests { margins, noise, max_depth, + trace_margins: &[], }; - evaluate( + let e = evaluate( &ids, &vecs, &queries, @@ -418,7 +471,9 @@ mod tests { &plan, None, ) - .unwrap() + .unwrap(); + assert!(e.traces.is_empty()); + (e.profile, e.reports) } fn fixed(reports: &[RescoreReport]) -> Vec<&RescoreReport> { @@ -508,4 +563,63 @@ mod tests { let (_, all) = run(&[&i8c], &[1000], &[], 0.0, 1000); assert_eq!(fixed(&all)[0].depth, 320); } + + #[test] + fn traces_replay_the_fixed_and_margin_cells() { + let (ids, vecs) = clustered(); + let queries = self_queries(&ids, &vecs, 30); + let f32c = F32Codec::new(64); + let noise = profile_noise(); + let margins = [1.0, 2.0]; + let plan = Plan { + seconds: &[&f32c], + depths: &[10, 20, 40], + margins: &margins, + noise, + max_depth: 200, + trace_margins: &margins, + }; + let e = evaluate( + &ids, + &vecs, + &queries, + &BinaryCodec::new(64), + Metric::Cosine, + 10, + &plan, + None, + ) + .unwrap(); + assert_eq!(e.traces.len(), 30); + let replay = |depth_of: &dyn Fn(&QueryTrace) -> usize| -> f64 { + e.traces + .iter() + .map(|t| t.ranks.iter().filter(|&&r| r <= depth_of(t)).count() as f64 / 10.0) + .sum::() + / 30.0 + }; + for r in &e.reports { + let replayed = match r.policy { + Policy::None => replay(&|_| 10), + Policy::Fixed { depth } => replay(&|_| depth), + Policy::Margin { sigmas } => { + let m = margins.iter().position(|&x| x == sigmas).unwrap(); + let mean: f64 = e.traces.iter().map(|t| t.margin_depths[m] as f64).sum::() / 30.0; + assert!((mean - r.mean_depth).abs() < 1e-9); + replay(&|t| t.margin_depths[m]) + } + }; + assert!( + (replayed - r.recall_at_k).abs() < 1e-9, + "{:?}: {replayed} vs {}", + r.policy, + r.recall_at_k + ); + } + assert!( + e.traces + .iter() + .all(|t| t.band > 0.0 && t.band_hat > 0.0 && t.ranks.len() == 10) + ); + } } diff --git a/crates/hvec-core/src/codecs/hamming.rs b/crates/hvec-core/src/codecs/hamming.rs new file mode 100644 index 0000000..ba81e98 --- /dev/null +++ b/crates/hvec-core/src/codecs/hamming.rs @@ -0,0 +1,191 @@ +//! Symmetric binary scoring: sign bits on both sides, compared by popcount. +//! +//! `binary-hamming` stores exactly what `binary` stores, one sign bit per +//! dimension. The difference is the query. `binary` scores the f32 query +//! against the bits (asymmetric: the query keeps its magnitudes). +//! `binary-hamming` binarises the query too, once per search in +//! [`Codec::prepare_query`], and scores each stored vector with XOR and +//! popcount over 64-bit words. That is the form whose cost is set by the bytes +//! read, so it is the binary codec a speed claim can be made for. +//! +//! Error model: approximate, and coarser than `binary`, because both sides are +//! quantized. For sign random-projection codes the fraction of differing bits +//! estimates the angle between the vectors divided by π (Charikar 2002), so +//! the score is `cos(π · hamming / d)`, an estimate of cosine similarity. +//! `Dot` is treated as cosine (embeddings are unit length); `L2` is the +//! unit-vector distance `2 − 2·cos`. Unlike every other codec, `score` does +//! not equal the metric on the decoded vector, because the query is quantized +//! as well; tests check it against the Hamming distance instead. + +use crate::codecs::{check_dim, check_payload}; +use crate::{Codec, CodecError, Encoded, Metric, Vector}; + +/// Sign bits, scored symmetrically by Hamming distance. +#[derive(Debug, Clone)] +pub struct HammingCodec { + dimension: usize, +} + +impl HammingCodec { + #[must_use] + pub fn new(dimension: usize) -> Self { + Self { dimension } + } + + fn pack(&self, v: &[f32]) -> Vec { + let mut out = vec![0u8; self.encoded_size()]; + for (i, &x) in v.iter().enumerate() { + if x > 0.0 { + out[i / 8] |= 1 << (i % 8); + } + } + out + } + + /// Number of 64-bit words a code occupies, the last one zero-padded. + fn words(&self) -> usize { + self.encoded_size().div_ceil(8) + } + + fn word(bytes: &[u8], j: usize) -> u64 { + let start = j * 8; + let end = (start + 8).min(bytes.len()); + let mut w = [0u8; 8]; + w[..end - start].copy_from_slice(&bytes[start..end]); + u64::from_le_bytes(w) + } + + /// Hamming distance between a prepared query and a stored code. + fn hamming(&self, prepared: &[f32], encoded: &[u8]) -> u32 { + let mut h = 0u32; + if encoded.len() % 8 == 0 { + for (j, chunk) in encoded.chunks_exact(8).enumerate() { + let x = u64::from_le_bytes(chunk.try_into().unwrap_or([0; 8])); + h += (x ^ Self::query_word(prepared, j)).count_ones(); + } + } else { + for j in 0..self.words() { + h += (Self::word(encoded, j) ^ Self::query_word(prepared, j)).count_ones(); + } + } + h + } + + /// The prepared query holds its packed sign bits, 32 per f32 slot (the + /// slot's bit pattern, not a number), two slots per 64-bit word. + fn query_word(prepared: &[f32], j: usize) -> u64 { + u64::from(prepared[2 * j].to_bits()) | (u64::from(prepared[2 * j + 1].to_bits()) << 32) + } +} + +impl Codec for HammingCodec { + fn name(&self) -> &str { + "binary-hamming" + } + + fn dimension(&self) -> usize { + self.dimension + } + + fn encoded_size(&self) -> usize { + self.dimension.div_ceil(8) + } + + fn supported_metrics(&self) -> &'static [Metric] { + &[Metric::Cosine, Metric::Dot, Metric::L2] + } + + fn encode(&self, vector: &[f32]) -> Result { + check_dim(self.dimension, vector.len())?; + Ok(Encoded(self.pack(vector))) + } + + /// The same reconstruction as `binary`: ±1/√d per dimension. + fn decode(&self, encoded: &[u8]) -> Result { + check_payload(self.encoded_size(), encoded.len())?; + let mag = (1.0 / (self.dimension as f64).sqrt()) as f32; + Ok((0..self.dimension) + .map(|i| if encoded[i / 8] >> (i % 8) & 1 == 1 { mag } else { -mag }) + .collect()) + } + + fn score(&self, query: &[f32], encoded: &[u8], metric: Metric) -> Result { + let prepared = self.prepare_query(query)?; + self.score_prepared(&prepared, encoded, metric) + } + + /// Binarise and pack the query once per search. + fn prepare_query(&self, query: &[f32]) -> Result { + check_dim(self.dimension, query.len())?; + let mut bytes = self.pack(query); + bytes.resize(self.words() * 8, 0); + Ok(bytes + .chunks_exact(4) + .map(|c| f32::from_bits(u32::from_le_bytes([c[0], c[1], c[2], c[3]]))) + .collect()) + } + + fn score_prepared(&self, prepared: &[f32], encoded: &[u8], metric: Metric) -> Result { + check_payload(self.encoded_size(), encoded.len())?; + if prepared.len() != self.words() * 2 { + return Err(CodecError::DimensionMismatch { + expected: self.words() * 2, + got: prepared.len(), + }); + } + let h = self.hamming(prepared, encoded); + let cos = (std::f64::consts::PI * f64::from(h) / self.dimension as f64).cos(); + Ok(match metric { + Metric::Cosine | Metric::Dot => cos as f32, + Metric::L2 => (2.0 - 2.0 * cos) as f32, + }) + } +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::codec_by_name; + + fn naive_hamming(a: &[f32], b: &[f32]) -> u32 { + a.iter().zip(b).filter(|(x, y)| (**x > 0.0) != (**y > 0.0)).count() as u32 + } + + #[test] + fn popcount_equals_the_naive_hamming_distance() { + for dim in [384, 100, 7, 64, 65] { + let c = HammingCodec::new(dim); + let a: Vec = (0..dim).map(|i| ((i * 37 % 11) as f32) - 5.0).collect(); + let b: Vec = (0..dim).map(|i| ((i * 13 % 7) as f32) - 3.0).collect(); + let p = c.prepare_query(&a).unwrap(); + let e = c.encode(&b).unwrap(); + assert_eq!(c.hamming(&p, &e), naive_hamming(&a, &b), "dim {dim}"); + let want = (std::f64::consts::PI * f64::from(naive_hamming(&a, &b)) / dim as f64).cos() as f32; + assert_eq!(c.score(&a, &e, Metric::Cosine).unwrap(), want); + assert_eq!(c.score_prepared(&p, &e, Metric::Cosine).unwrap(), want); + } + } + + #[test] + fn identical_signs_score_one_and_opposite_minus_one() { + let c = HammingCodec::new(16); + let v: Vec = (0..16).map(|i| if i % 3 == 0 { 1.0 } else { -1.0 }).collect(); + let neg: Vec = v.iter().map(|x| -x).collect(); + assert_eq!(c.score(&v, &c.encode(&v).unwrap(), Metric::Cosine).unwrap(), 1.0); + assert!((c.score(&v, &c.encode(&neg).unwrap(), Metric::Cosine).unwrap() + 1.0).abs() < 1e-6); + assert!(c.score(&v, &c.encode(&v).unwrap(), Metric::L2).unwrap().abs() < 1e-6); + } + + #[test] + fn stores_the_same_bits_as_binary_and_is_built_by_name() { + let c = codec_by_name("binary-hamming", 24).unwrap(); + let b = codec_by_name("binary", 24).unwrap(); + let v: Vec = (0..24).map(|i| (i as f32 * 0.7).sin()).collect(); + assert_eq!(c.encode(&v).unwrap(), b.encode(&v).unwrap()); + assert_eq!(c.encoded_size(), 3); + assert!( + c.score_prepared(&[0.0; 3], &c.encode(&v).unwrap(), Metric::Cosine) + .is_err() + ); + } +} diff --git a/crates/hvec-core/src/codecs/mixed.rs b/crates/hvec-core/src/codecs/mixed.rs index fda7b35..5d2a476 100644 --- a/crates/hvec-core/src/codecs/mixed.rs +++ b/crates/hvec-core/src/codecs/mixed.rs @@ -111,8 +111,8 @@ impl MixedCodec { } /// Calls `f(dimension, reconstructed value)` for every dimension. - fn for_each_value(fit: &Fitted, encoded: &Encoded, mut f: impl FnMut(usize, f64)) { - let mut reader = BitReader::new(&encoded.0); + fn for_each_value(fit: &Fitted, encoded: &[u8], mut f: impl FnMut(usize, f64)) { + let mut reader = BitReader::new(encoded); for (i, &b) in fit.bits.iter().enumerate() { let m = f64::from(fit.mean[i]); if b == 0 { @@ -447,8 +447,8 @@ impl Codec for MixedCodec { Ok(Encoded(w.finish(self.bytes))) } - fn decode(&self, encoded: &Encoded) -> Result { - check_payload(self.bytes, encoded.0.len())?; + fn decode(&self, encoded: &[u8]) -> Result { + check_payload(self.bytes, encoded.len())?; let f = self.state()?; let mut y = vec![0f32; self.dimension]; Self::for_each_value(f, encoded, |i, v| y[i] = v as f32); @@ -468,7 +468,7 @@ impl Codec for MixedCodec { }) } - fn score(&self, query: &[f32], encoded: &Encoded, metric: Metric) -> Result { + fn score(&self, query: &[f32], encoded: &[u8], metric: Metric) -> Result { let prepared = self.prepare_query(query)?; self.score_prepared(&prepared, encoded, metric) } @@ -478,9 +478,9 @@ impl Codec for MixedCodec { Ok(Self::rotate(self.state()?, query)) } - fn score_prepared(&self, prepared: &[f32], encoded: &Encoded, metric: Metric) -> Result { + fn score_prepared(&self, prepared: &[f32], encoded: &[u8], metric: Metric) -> Result { check_dim(self.dimension, prepared.len())?; - check_payload(self.bytes, encoded.0.len())?; + check_payload(self.bytes, encoded.len())?; let f = self.state()?; let (mut dot, mut stored_sq, mut query_sq) = (0f64, 0f64, 0f64); Self::for_each_value(f, encoded, |i, v| { diff --git a/crates/hvec-core/src/codecs/mod.rs b/crates/hvec-core/src/codecs/mod.rs index e8eec2f..fa2e408 100644 --- a/crates/hvec-core/src/codecs/mod.rs +++ b/crates/hvec-core/src/codecs/mod.rs @@ -5,15 +5,17 @@ //! approximate the reference distance on the original f32 vector within the //! codec's documented error model. +mod hamming; mod mixed; mod pq; mod truncate; +pub use hamming::HammingCodec; pub use mixed::MixedCodec; pub use pq::PqCodec; pub use truncate::TruncateCodec; -use crate::{Codec, CodecError, Encoded, Metric, Vector, distance}; +use crate::{Codec, CodecError, Encoded, Metric, Vector}; pub(crate) fn check_dim(expected: usize, got: usize) -> Result<(), CodecError> { if got == expected { @@ -73,20 +75,38 @@ impl Codec for F32Codec { Ok(Encoded(vector.iter().flat_map(|v| v.to_le_bytes()).collect())) } - fn decode(&self, encoded: &Encoded) -> Result { - check_payload(self.encoded_size(), encoded.0.len())?; + fn decode(&self, encoded: &[u8]) -> Result { + check_payload(self.encoded_size(), encoded.len())?; Ok(encoded - .0 .chunks_exact(4) .map(|c| f32::from_le_bytes([c[0], c[1], c[2], c[3]])) .collect()) } - fn score(&self, query: &[f32], encoded: &Encoded, metric: Metric) -> Result { + /// Reads the stored floats in place, in one fused pass: nothing is + /// allocated per vector. The three running sums accumulate in the same + /// order as [`distance::score`], so the result is identical to scoring the + /// decoded vector. + fn score(&self, query: &[f32], encoded: &[u8], metric: Metric) -> Result { check_dim(self.dimension, query.len())?; - // Baseline: decoding is the documented behaviour here. - let stored = self.decode(encoded)?; - Ok(distance::score(query, &stored, metric)) + check_payload(self.encoded_size(), encoded.len())?; + let stored = encoded + .chunks_exact(4) + .map(|c| f32::from_le_bytes([c[0], c[1], c[2], c[3]])); + Ok(match metric { + Metric::Dot => query.iter().zip(stored).map(|(q, x)| q * x).sum(), + Metric::Cosine => { + let (mut dot, mut qq, mut xx) = (0f32, 0f32, 0f32); + for (&q, x) in query.iter().zip(stored) { + dot += q * x; + qq += q * q; + xx += x * x; + } + let denom = qq.sqrt() * xx.sqrt(); + if denom == 0.0 { 0.0 } else { dot / denom } + } + Metric::L2 => query.iter().zip(stored).map(|(q, x)| (q - x) * (q - x)).sum(), + }) } } @@ -159,16 +179,16 @@ impl Codec for Int8Codec { Ok(Encoded(out)) } - fn decode(&self, encoded: &Encoded) -> Result { - check_payload(self.encoded_size(), encoded.0.len())?; - let (min, scale, q) = Self::split(&encoded.0); + fn decode(&self, encoded: &[u8]) -> Result { + check_payload(self.encoded_size(), encoded.len())?; + let (min, scale, q) = Self::split(encoded); Ok(q.iter().map(|&b| min + scale * f32::from(b)).collect()) } - fn score(&self, query: &[f32], encoded: &Encoded, metric: Metric) -> Result { + fn score(&self, query: &[f32], encoded: &[u8], metric: Metric) -> Result { check_dim(self.dimension, query.len())?; - check_payload(self.encoded_size(), encoded.0.len())?; - let (min, scale, q) = Self::split(&encoded.0); + check_payload(self.encoded_size(), encoded.len())?; + let (min, scale, q) = Self::split(encoded); let (min, scale) = (f64::from(min), f64::from(scale)); // One pass: everything below is derived from these five sums. @@ -257,27 +277,21 @@ impl Codec for BinaryCodec { Ok(Encoded(out)) } - fn decode(&self, encoded: &Encoded) -> Result { - check_payload(self.encoded_size(), encoded.0.len())?; + fn decode(&self, encoded: &[u8]) -> Result { + check_payload(self.encoded_size(), encoded.len())?; let mag = self.inv_sqrt_d() as f32; Ok((0..self.dimension) - .map(|i| { - if encoded.0[i / 8] >> (i % 8) & 1 == 1 { - mag - } else { - -mag - } - }) + .map(|i| if encoded[i / 8] >> (i % 8) & 1 == 1 { mag } else { -mag }) .collect()) } - fn score(&self, query: &[f32], encoded: &Encoded, metric: Metric) -> Result { + fn score(&self, query: &[f32], encoded: &[u8], metric: Metric) -> Result { check_dim(self.dimension, query.len())?; - check_payload(self.encoded_size(), encoded.0.len())?; + check_payload(self.encoded_size(), encoded.len())?; let (mut signed, mut sum_xx) = (0f64, 0f64); for (i, &x) in query.iter().enumerate() { let x = f64::from(x); - let positive = encoded.0[i / 8] >> (i % 8) & 1 == 1; + let positive = encoded[i / 8] >> (i % 8) & 1 == 1; signed += if positive { x } else { -x }; sum_xx += x * x; } @@ -474,14 +488,14 @@ impl Codec for BinaryCentredCodec { Ok(Encoded(out)) } - fn decode(&self, encoded: &Encoded) -> Result { - check_payload(self.encoded_size(), encoded.0.len())?; + fn decode(&self, encoded: &[u8]) -> Result { + check_payload(self.encoded_size(), encoded.len())?; let c = self.centring()?; Ok(c.mean .iter() .enumerate() .map(|(i, &m)| { - if encoded.0[i / 8] >> (i % 8) & 1 == 1 { + if encoded[i / 8] >> (i % 8) & 1 == 1 { m + c.scale } else { m - c.scale @@ -490,16 +504,16 @@ impl Codec for BinaryCentredCodec { .collect()) } - fn score(&self, query: &[f32], encoded: &Encoded, metric: Metric) -> Result { + fn score(&self, query: &[f32], encoded: &[u8], metric: Metric) -> Result { check_dim(self.dimension, query.len())?; - check_payload(self.encoded_size(), encoded.0.len())?; + check_payload(self.encoded_size(), encoded.len())?; let c = self.centring()?; let alpha = f64::from(c.scale); // Running sums over one pass: q·μ, q·s, μ·s and ‖q‖², with s = ±1. let (mut dot_qm, mut dot_qs, mut dot_ms, mut sum_qq) = (0f64, 0f64, 0f64, 0f64); for (i, (&q, &m)) in query.iter().zip(&c.mean).enumerate() { let (q, m) = (f64::from(q), f64::from(m)); - let positive = encoded.0[i / 8] >> (i % 8) & 1 == 1; + let positive = encoded[i / 8] >> (i % 8) & 1 == 1; dot_qm += q * m; sum_qq += q * q; if positive { @@ -528,6 +542,24 @@ impl Codec for BinaryCentredCodec { #[cfg(test)] mod tests { use super::*; + use crate::distance; + + #[test] + fn f32_scores_in_place_identically_to_the_reference() { + let mut rng = XorShift(31); + let c = F32Codec::new(384); + for _ in 0..50 { + let q = rng.vector(384); + let v = rng.vector(384); + let e = c.encode(&v).unwrap(); + for m in [Metric::Dot, Metric::Cosine, Metric::L2] { + assert_eq!(c.score(&q, &e, m).unwrap(), distance::score(&q, &v, m), "{m:?}"); + } + // The fused cosine matches the three-pass formula bit for bit. + let three_pass = distance::dot(&q, &v) / (distance::norm(&q) * distance::norm(&v)); + assert_eq!(distance::cosine(&q, &v), three_pass); + } + } /// Small deterministic generator so tests need no external crate. struct XorShift(u64); diff --git a/crates/hvec-core/src/codecs/pq.rs b/crates/hvec-core/src/codecs/pq.rs index d3bd372..b474f48 100644 --- a/crates/hvec-core/src/codecs/pq.rs +++ b/crates/hvec-core/src/codecs/pq.rs @@ -249,11 +249,11 @@ impl Codec for PqCodec { Ok(Encoded(codes)) } - fn decode(&self, encoded: &Encoded) -> Result { - check_payload(self.encoded_size(), encoded.0.len())?; + fn decode(&self, encoded: &[u8]) -> Result { + check_payload(self.encoded_size(), encoded.len())?; let books = self.books()?; let mut out = Vec::with_capacity(self.dimension); - for (j, &code) in encoded.0.iter().enumerate() { + for (j, &code) in encoded.iter().enumerate() { let c = usize::from(code); if c >= books.k { return Err(CodecError::BadParams { @@ -266,12 +266,12 @@ impl Codec for PqCodec { Ok(out) } - fn score(&self, query: &[f32], encoded: &Encoded, metric: Metric) -> Result { + fn score(&self, query: &[f32], encoded: &[u8], metric: Metric) -> Result { check_dim(self.dimension, query.len())?; - check_payload(self.encoded_size(), encoded.0.len())?; + check_payload(self.encoded_size(), encoded.len())?; let books = self.books()?; let (mut dot, mut stored_norm_sq, mut sum_qq) = (0f64, 0f64, 0f64); - for (j, &code) in encoded.0.iter().enumerate() { + for (j, &code) in encoded.iter().enumerate() { let c = usize::from(code); if c >= books.k { return Err(CodecError::BadParams { diff --git a/crates/hvec-core/src/codecs/truncate.rs b/crates/hvec-core/src/codecs/truncate.rs index 6c5398c..5f1bc22 100644 --- a/crates/hvec-core/src/codecs/truncate.rs +++ b/crates/hvec-core/src/codecs/truncate.rs @@ -110,13 +110,13 @@ impl Codec for TruncateCodec { self.inner.encode(self.prefix(vector)?) } - fn decode(&self, encoded: &Encoded) -> Result { + fn decode(&self, encoded: &[u8]) -> Result { let mut v = self.inner.decode(encoded)?; v.resize(self.dimension, 0.0); Ok(v) } - fn score(&self, query: &[f32], encoded: &Encoded, metric: Metric) -> Result { + fn score(&self, query: &[f32], encoded: &[u8], metric: Metric) -> Result { self.inner.score(self.prefix(query)?, encoded, metric) } @@ -126,7 +126,7 @@ impl Codec for TruncateCodec { self.inner.prepare_query(self.prefix(query)?) } - fn score_prepared(&self, prepared: &[f32], encoded: &Encoded, metric: Metric) -> Result { + fn score_prepared(&self, prepared: &[f32], encoded: &[u8], metric: Metric) -> Result { self.inner.score_prepared(prepared, encoded, metric) } } diff --git a/crates/hvec-core/src/distance.rs b/crates/hvec-core/src/distance.rs index a08d7cb..6b406b1 100644 --- a/crates/hvec-core/src/distance.rs +++ b/crates/hvec-core/src/distance.rs @@ -18,8 +18,18 @@ pub fn norm(a: &[f32]) -> f32 { /// Cosine similarity. Returns 0 when either vector has zero norm. #[must_use] pub fn cosine(a: &[f32], b: &[f32]) -> f32 { - let denom = norm(a) * norm(b); - if denom == 0.0 { 0.0 } else { dot(a, b) / denom } + debug_assert_eq!(a.len(), b.len()); + // One pass over both vectors. Each of the three sums accumulates in the + // same order as `dot` and `norm` would, so the result is identical to + // `dot(a, b) / (norm(a) * norm(b))`, at a third of the memory traffic. + let (mut ab, mut aa, mut bb) = (0f32, 0f32, 0f32); + for (&x, &y) in a.iter().zip(b) { + ab += x * y; + aa += x * x; + bb += y * y; + } + let denom = aa.sqrt() * bb.sqrt(); + if denom == 0.0 { 0.0 } else { ab / denom } } /// Squared Euclidean distance. diff --git a/crates/hvec-core/src/lib.rs b/crates/hvec-core/src/lib.rs index b097b3a..d637573 100644 --- a/crates/hvec-core/src/lib.rs +++ b/crates/hvec-core/src/lib.rs @@ -82,9 +82,27 @@ pub enum CodecError { } /// A compressed representation of a vector, as stored on disk. +/// +/// This is the owned form [`Codec::encode`] returns. Scoring and decoding +/// borrow the bytes instead (`&[u8]`), so a codec can score codes in place: +/// in a contiguous buffer, a memory-mapped file or a database page. `Encoded` +/// dereferences to `[u8]`, so `&encoded` works wherever `&[u8]` is expected. #[derive(Debug, Clone, PartialEq, Eq)] pub struct Encoded(pub Vec); +impl std::ops::Deref for Encoded { + type Target = [u8]; + fn deref(&self) -> &[u8] { + &self.0 + } +} + +impl AsRef<[u8]> for Encoded { + fn as_ref(&self) -> &[u8] { + &self.0 + } +} + /// A compression scheme whose compressed form supports distance computation. /// /// Implementations must document their error model: exact, bounded, or @@ -108,13 +126,13 @@ pub trait Codec: Send + Sync + std::fmt::Debug { fn encode(&self, vector: &[f32]) -> Result; /// Reconstruct an approximation of the original vector. - fn decode(&self, encoded: &Encoded) -> Result; + fn decode(&self, encoded: &[u8]) -> Result; /// Score an uncompressed query against a compressed stored vector. /// /// This is the asymmetric form. It must not decode `encoded` into a full /// `Vector` unless the codec documents that it is a baseline. - fn score(&self, query: &[f32], encoded: &Encoded, metric: Metric) -> Result; + fn score(&self, query: &[f32], encoded: &[u8], metric: Metric) -> Result; /// Transform a query once per search, before it is scored against many /// stored vectors with [`Codec::score_prepared`]. @@ -131,7 +149,7 @@ pub trait Codec: Send + Sync + std::fmt::Debug { /// `score(query, …)` for the original query. The default forwards to /// [`Codec::score`], which is right whenever `prepare_query` is the /// identity. - fn score_prepared(&self, prepared: &[f32], encoded: &Encoded, metric: Metric) -> Result { + fn score_prepared(&self, prepared: &[f32], encoded: &[u8], metric: Metric) -> Result { self.score(prepared, encoded, metric) } @@ -184,7 +202,8 @@ pub trait Codec: Send + Sync + std::fmt::Debug { /// Instantiate a codec by its stable name, unfitted. /// -/// Names: `f32`, `int8`, `binary`, `binary-centred`, `pq` (product +/// Names: `f32`, `int8`, `binary`, `binary-centred`, `binary-hamming` (the +/// same bits as `binary`, scored symmetrically by popcount), `pq` (product /// quantization with `m` subspaces; bare `pq` means one byte per 8 /// dimensions), `mrl` (keep the first `k` dimensions, Matryoshka style) /// `mrl+` (truncate, then encode with another codec, for @@ -200,6 +219,7 @@ pub fn codec_by_name(name: &str, dimension: usize) -> Result, Cod "int8" => Ok(Box::new(codecs::Int8Codec::new(dimension))), "binary" => Ok(Box::new(codecs::BinaryCodec::new(dimension))), "binary-centred" => Ok(Box::new(codecs::BinaryCentredCodec::new(dimension))), + "binary-hamming" => Ok(Box::new(codecs::HammingCodec::new(dimension))), _ => { if let Some(rest) = name.strip_prefix("pq") { let m = parse_param(name, rest, Some(dimension / 8))?; @@ -276,6 +296,7 @@ pub fn codec_names() -> &'static [&'static str] { "int8", "binary", "binary-centred", + "binary-hamming", "pq", "mrl", "mrl+", diff --git a/crates/hvec-store/src/codes.rs b/crates/hvec-store/src/codes.rs new file mode 100644 index 0000000..3454615 --- /dev/null +++ b/crates/hvec-store/src/codes.rs @@ -0,0 +1,393 @@ +//! A collection's codes in one contiguous buffer, scanned in place. +//! +//! The SQLite table is the system of record, one row per chunk. Scanning it +//! row by row on every query copies every blob (and, for search, every +//! chunk's text) out of the database. [`CodeBuffer`] is the columnar form: +//! the codes back to back in a single allocation, with ids and sources as side +//! arrays. Scoring borrows each code as a `&[u8]` slice of the buffer, so +//! nothing is copied or allocated per vector, and the text is read only for +//! the hits. +//! +//! A buffer can be written to a flat file and read back with one read. A +//! memory-mapped file would avoid even that copy, but needs `unsafe` (the OS +//! may change a mapped file underneath safe code), which this workspace +//! forbids; SQLite's own memory map is available instead through +//! [`Store::set_mmap_size`](crate::Store::set_mmap_size). + +use std::cmp::Ordering; +use std::collections::BinaryHeap; +use std::path::Path; + +use hvec_core::{Codec, CodecError, Metric}; + +const MAGIC: &[u8; 8] = b"HVECODE1"; + +/// Errors reading or writing a code file. +#[derive(Debug, thiserror::Error)] +#[non_exhaustive] +pub enum CodeFileError { + #[error("could not read or write {path}")] + Io { + path: std::path::PathBuf, + #[source] + source: std::io::Error, + }, + #[error("{path} is not an hvec code file or is truncated")] + Format { path: std::path::PathBuf }, +} + +/// One collection's codes, back to back, with their chunk ids and sources. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct CodeBuffer { + code_size: usize, + ids: Vec, + sources: Vec, + codes: Vec, +} + +/// A scored position in a [`CodeBuffer`]. +#[derive(Debug, Clone, Copy, PartialEq)] +pub struct Scored { + /// Index into the buffer. + pub index: usize, + pub score: f32, +} + +/// Heap entry ordered so that the *worst* kept hit is on top. +#[derive(Debug, Clone, Copy)] +struct Entry { + key: f32, + index: usize, +} + +impl PartialEq for Entry { + fn eq(&self, other: &Self) -> bool { + self.cmp(other) == Ordering::Equal + } +} +impl Eq for Entry {} +impl PartialOrd for Entry { + fn partial_cmp(&self, other: &Self) -> Option { + Some(self.cmp(other)) + } +} +impl Ord for Entry { + // `key` is "higher is better" after orienting by the metric. The heap's + // top must be the worst: lowest key, and among equal keys the highest + // index, so earlier vectors win ties as they did in a stable sort. + fn cmp(&self, other: &Self) -> Ordering { + other.key.total_cmp(&self.key).then(self.index.cmp(&other.index)) + } +} + +impl CodeBuffer { + /// An empty buffer for codes of `code_size` bytes. + #[must_use] + pub fn new(code_size: usize) -> Self { + Self { + code_size, + ids: Vec::new(), + sources: Vec::new(), + codes: Vec::new(), + } + } + + /// Append one code. Panics if its length is not the buffer's code size. + pub fn push(&mut self, id: i64, source: String, code: &[u8]) { + assert_eq!(code.len(), self.code_size, "code size mismatch"); + self.ids.push(id); + self.sources.push(source); + self.codes.extend_from_slice(code); + } + + /// Re-encode f32 vectors with `codec` into a new buffer. + pub fn encode( + codec: &dyn Codec, + ids: &[i64], + sources: &[String], + vectors: &[Vec], + ) -> Result { + let mut b = Self::new(codec.encoded_size()); + b.codes.reserve(vectors.len() * b.code_size); + for ((id, source), v) in ids.iter().zip(sources).zip(vectors) { + b.push(*id, source.clone(), &codec.encode(v)?); + } + Ok(b) + } + + #[must_use] + pub fn len(&self) -> usize { + self.ids.len() + } + + #[must_use] + pub fn is_empty(&self) -> bool { + self.ids.is_empty() + } + + #[must_use] + pub fn code_size(&self) -> usize { + self.code_size + } + + #[must_use] + pub fn ids(&self) -> &[i64] { + &self.ids + } + + #[must_use] + pub fn sources(&self) -> &[String] { + &self.sources + } + + /// The whole code region, `len() × code_size()` bytes. + #[must_use] + pub fn bytes(&self) -> &[u8] { + &self.codes + } + + /// The code at `index`, borrowed from the buffer. + #[must_use] + pub fn code(&self, index: usize) -> &[u8] { + &self.codes[index * self.code_size..(index + 1) * self.code_size] + } + + /// Score every code and keep the best `k`, best first. The query is + /// prepared once; each code is scored in place. + pub fn search( + &self, + query: &[f32], + k: usize, + codec: &dyn Codec, + metric: Metric, + ) -> Result, CodecError> { + self.search_threads(query, k, codec, metric, 1) + } + + /// As [`CodeBuffer::search`], with the scan split over `threads` scoped + /// threads. Results are identical to the single-threaded scan. + pub fn search_threads( + &self, + query: &[f32], + k: usize, + codec: &dyn Codec, + metric: Metric, + threads: usize, + ) -> Result, CodecError> { + if codec.encoded_size() != self.code_size { + return Err(CodecError::PayloadSize { + expected: codec.encoded_size(), + got: self.code_size, + }); + } + let prepared = codec.prepare_query(query)?; + let higher = metric.higher_is_better(); + let n = self.len(); + let threads = threads.clamp(1, n.max(1)); + let scan = |lo: usize, hi: usize| -> Result, CodecError> { + let mut heap = BinaryHeap::with_capacity(k + 1); + if k == 0 { + return Ok(heap); + } + for (offset, code) in self.codes[lo * self.code_size..hi * self.code_size] + .chunks_exact(self.code_size) + .enumerate() + { + let s = codec.score_prepared(&prepared, code, metric)?; + let e = Entry { + key: if higher { s } else { -s }, + index: lo + offset, + }; + if heap.len() < k { + heap.push(e); + } else if heap.peek().is_some_and(|worst| e < *worst) { + heap.pop(); + heap.push(e); + } + } + Ok(heap) + }; + let heaps: Vec> = if threads == 1 { + vec![scan(0, n)?] + } else { + let per = n.div_ceil(threads); + std::thread::scope(|s| { + let handles: Vec<_> = (0..threads) + .map(|t| { + let (lo, hi) = ((t * per).min(n), ((t + 1) * per).min(n)); + s.spawn(move || scan(lo, hi)) + }) + .collect(); + handles + .into_iter() + .map(|h| h.join().expect("scan thread panicked")) + .collect::, _>>() + })? + }; + let mut all: Vec = heaps.into_iter().flat_map(BinaryHeap::into_vec).collect(); + all.sort(); + all.truncate(k); + Ok(all + .into_iter() + .map(|e| Scored { + index: e.index, + score: if higher { e.key } else { -e.key }, + }) + .collect()) + } + + /// Write the buffer as a flat file: a header, the ids, then the codes. + /// Sources are not written; they stay in the database. + pub fn write_file(&self, path: &Path) -> Result<(), CodeFileError> { + let mut out = Vec::with_capacity(24 + self.len() * (8 + self.code_size)); + out.extend_from_slice(MAGIC); + out.extend_from_slice(&(self.code_size as u64).to_le_bytes()); + out.extend_from_slice(&(self.len() as u64).to_le_bytes()); + for id in &self.ids { + out.extend_from_slice(&id.to_le_bytes()); + } + out.extend_from_slice(&self.codes); + std::fs::write(path, out).map_err(|source| CodeFileError::Io { + path: path.to_path_buf(), + source, + }) + } + + /// Read a file written by [`CodeBuffer::write_file`] with a single read. + /// Sources come back empty. + pub fn read_file(path: &Path) -> Result { + let data = std::fs::read(path).map_err(|source| CodeFileError::Io { + path: path.to_path_buf(), + source, + })?; + let bad = || CodeFileError::Format { + path: path.to_path_buf(), + }; + if data.len() < 24 || &data[..8] != MAGIC { + return Err(bad()); + } + let u64_at = |o: usize| u64::from_le_bytes(data[o..o + 8].try_into().unwrap_or([0; 8])); + let code_size = usize::try_from(u64_at(8)).map_err(|_| bad())?; + let n = usize::try_from(u64_at(16)).map_err(|_| bad())?; + let ids_end = 24usize.checked_add(n.checked_mul(8).ok_or_else(bad)?).ok_or_else(bad)?; + let end = ids_end + .checked_add(n.checked_mul(code_size).ok_or_else(bad)?) + .ok_or_else(bad)?; + if data.len() != end { + return Err(bad()); + } + let ids = data[24..ids_end] + .chunks_exact(8) + .map(|c| i64::from_le_bytes(c.try_into().unwrap_or([0; 8]))) + .collect(); + let mut codes = data; + codes.drain(..ids_end); + Ok(Self { + code_size, + ids, + sources: vec![String::new(); n], + codes, + }) + } +} + +#[cfg(test)] +mod tests { + use super::*; + use hvec_core::codecs::{BinaryCodec, F32Codec, HammingCodec, Int8Codec}; + use hvec_core::distance; + + fn corpus(n: usize, dim: usize) -> Vec> { + let mut s = 12345u64; + (0..n) + .map(|_| { + let mut v: Vec = (0..dim) + .map(|_| { + s ^= s << 13; + s ^= s >> 7; + s ^= s << 17; + ((s >> 40) as f32 / (1u64 << 24) as f32) * 2.0 - 1.0 + }) + .collect(); + distance::normalize(&mut v); + v + }) + .collect() + } + + fn reference(codec: &dyn Codec, q: &[f32], vs: &[Vec], k: usize, metric: Metric) -> Vec<(usize, f32)> { + let mut all: Vec<(usize, f32)> = vs + .iter() + .enumerate() + .map(|(i, v)| (i, codec.score(q, &codec.encode(v).unwrap(), metric).unwrap())) + .collect(); + if metric.higher_is_better() { + all.sort_by(|a, b| b.1.total_cmp(&a.1).then(a.0.cmp(&b.0))); + } else { + all.sort_by(|a, b| a.1.total_cmp(&b.1).then(a.0.cmp(&b.0))); + } + all.truncate(k); + all + } + + #[test] + fn buffer_search_matches_a_full_sort_for_every_codec_and_thread_count() { + let vs = corpus(500, 32); + let ids: Vec = (0..500).map(|i| i * 3 + 1).collect(); + let sources: Vec = (0..500).map(|i| format!("doc{i}.txt")).collect(); + let codecs: Vec> = vec![ + Box::new(F32Codec::new(32)), + Box::new(Int8Codec::new(32)), + Box::new(BinaryCodec::new(32)), + Box::new(HammingCodec::new(32)), + ]; + for codec in &codecs { + let buf = CodeBuffer::encode(codec.as_ref(), &ids, &sources, &vs).unwrap(); + assert_eq!(buf.len(), 500); + assert_eq!(buf.bytes().len(), 500 * codec.encoded_size()); + for metric in [Metric::Cosine, Metric::L2] { + let want = reference(codec.as_ref(), &vs[7], &vs, 10, metric); + for threads in [1, 3, 8] { + let got = buf.search_threads(&vs[7], 10, codec.as_ref(), metric, threads).unwrap(); + let got: Vec<(usize, f32)> = got.iter().map(|s| (s.index, s.score)).collect(); + assert_eq!(got, want, "{} {metric:?} threads={threads}", codec.name()); + } + } + } + } + + #[test] + fn ties_keep_the_earlier_vector_and_k_is_respected() { + let codec = BinaryCodec::new(8); + let v = vec![vec![1.0f32; 8]; 5]; + let ids: Vec = (0..5).collect(); + let buf = CodeBuffer::encode(&codec, &ids, &vec![String::new(); 5], &v).unwrap(); + let got = buf.search(&v[0], 3, &codec, Metric::Cosine).unwrap(); + assert_eq!(got.iter().map(|s| s.index).collect::>(), vec![0, 1, 2]); + assert!(buf.search(&v[0], 0, &codec, Metric::Cosine).unwrap().is_empty()); + assert_eq!(buf.search(&v[0], 99, &codec, Metric::Cosine).unwrap().len(), 5); + assert!(buf.search(&v[0], 3, &Int8Codec::new(8), Metric::Cosine).is_err()); + } + + #[test] + fn code_files_roundtrip_and_reject_damage() { + let vs = corpus(40, 16); + let ids: Vec = (100..140).collect(); + let codec = Int8Codec::new(16); + let buf = CodeBuffer::encode(&codec, &ids, &vec![String::new(); 40], &vs).unwrap(); + let dir = tempfile::tempdir().unwrap(); + let path = dir.path().join("c.codes"); + buf.write_file(&path).unwrap(); + let back = CodeBuffer::read_file(&path).unwrap(); + assert_eq!(back, buf); + let mut data = std::fs::read(&path).unwrap(); + data.pop(); + std::fs::write(&path, &data).unwrap(); + assert!(matches!( + CodeBuffer::read_file(&path), + Err(CodeFileError::Format { .. }) + )); + std::fs::write(&path, b"not a code file at all.").unwrap(); + assert!(CodeBuffer::read_file(&path).is_err()); + } +} diff --git a/crates/hvec-store/src/lib.rs b/crates/hvec-store/src/lib.rs index 8348247..cac8128 100644 --- a/crates/hvec-store/src/lib.rs +++ b/crates/hvec-store/src/lib.rs @@ -6,15 +6,20 @@ //! by brute-force scan. That is deliberate: the point of hvec is to measure //! the codec, not an index. +mod codes; mod runs; mod schema; mod sessions; mod vectors; +use std::cell::RefCell; +use std::collections::HashMap; use std::path::Path; +use std::sync::Arc; use rusqlite::Connection; +pub use codes::{CodeBuffer, CodeFileError, Scored}; pub use runs::RunRow; pub use sessions::{Session, StoredMessage}; pub use vectors::{Collection, Hit, NewChunk, NewCollection}; @@ -62,6 +67,17 @@ pub enum StoreError { #[derive(Debug)] pub struct Store { conn: Connection, + /// Columnar copies of collections' codes, built on first search and + /// reused until the collection changes. Dropped on this handle's own + /// writes, and wholesale when another connection commits (SQLite's + /// `data_version` moves). + codes: RefCell, +} + +#[derive(Debug, Default)] +struct CodeCache { + data_version: i64, + by_collection: HashMap>, } impl Store { @@ -90,7 +106,10 @@ impl Store { conn.execute_batch(alter)?; } } - Ok(Self { conn }) + Ok(Self { + conn, + codes: RefCell::new(CodeCache::default()), + }) } fn has_column(conn: &Connection, table: &str, column: &str) -> Result { diff --git a/crates/hvec-store/src/vectors.rs b/crates/hvec-store/src/vectors.rs index e36aca0..bca99bc 100644 --- a/crates/hvec-store/src/vectors.rs +++ b/crates/hvec-store/src/vectors.rs @@ -4,7 +4,9 @@ use hvec_core::{Codec, CodecError, Encoded, Metric}; use rusqlite::{OptionalExtension, params}; use serde::{Deserialize, Serialize}; -use crate::{Result, Store, StoreError}; +use std::sync::Arc; + +use crate::{CodeBuffer, Result, Store, StoreError}; /// Metadata for a collection. #[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] @@ -140,6 +142,7 @@ impl Store { if n == 0 { return Err(StoreError::NoSuchCollection(name.to_owned())); } + self.forget_codes(name); Ok(()) } @@ -156,9 +159,68 @@ impl Store { } } tx.commit()?; + self.forget_codes(collection); Ok(chunks.len()) } + /// Drop the cached code buffer for a collection after a write. + fn forget_codes(&self, collection: &str) { + self.codes.borrow_mut().by_collection.remove(collection); + } + + /// Read a collection's codes into one contiguous buffer, in id order. + /// Each blob is borrowed from SQLite's row and appended, so the whole + /// collection costs one copy, not one allocation per vector. + pub fn load_codes(&self, collection: &str) -> Result { + let c = self.require_collection(collection)?; + let size = c.codec()?.encoded_size(); + let mut buf = CodeBuffer::new(size); + let mut stmt = self + .conn + .prepare_cached("SELECT id, source, vector FROM chunks WHERE collection = ?1 ORDER BY id")?; + let mut rows = stmt.query(params![collection])?; + while let Some(row) = rows.next()? { + let blob = row.get_ref(2)?.as_blob().map_err(rusqlite::Error::from)?; + if blob.len() != size { + return Err(StoreError::Codec(CodecError::PayloadSize { + expected: size, + got: blob.len(), + })); + } + buf.push(row.get(0)?, row.get(1)?, blob); + } + Ok(buf) + } + + /// The cached code buffer for a collection, loaded on first use. + pub fn codes(&self, collection: &str) -> Result> { + let version: i64 = self.conn.query_row("PRAGMA data_version", [], |r| r.get(0))?; + { + let mut cache = self.codes.borrow_mut(); + if cache.data_version != version { + cache.by_collection.clear(); + cache.data_version = version; + } + if let Some(b) = cache.by_collection.get(collection) { + return Ok(Arc::clone(b)); + } + } + let b = Arc::new(self.load_codes(collection)?); + self.codes + .borrow_mut() + .by_collection + .insert(collection.to_owned(), Arc::clone(&b)); + Ok(b) + } + + /// Ask SQLite to memory-map up to `bytes` of the database file, so + /// borrowed blobs are read from the OS page cache without a copy into + /// SQLite's own buffers. `0` turns it off. + pub fn set_mmap_size(&self, bytes: u64) -> Result<()> { + self.conn.execute_batch(&format!("PRAGMA mmap_size = {bytes}"))?; + Ok(()) + } + /// Every stored chunk's id and encoded vector, in insertion order. /// /// Used by benchmarks that re-encode a collection with other codecs. @@ -205,7 +267,8 @@ impl Store { /// Brute-force top-k search scored through the codec. /// /// Scores every stored vector against `query` in the compressed domain, - /// then keeps the best `k` according to the metric's direction. + /// in place in the collection's cached [`CodeBuffer`], then reads source, + /// ordinal and text for the best `k` only. pub fn search( &self, collection: &str, @@ -213,6 +276,71 @@ impl Store { k: usize, codec: &dyn Codec, metric: Metric, + ) -> Result> { + let buf = self.codes(collection)?; + let top = buf.search(query, k, codec, metric)?; + let mut stmt = self + .conn + .prepare_cached("SELECT source, ordinal, text FROM chunks WHERE id = ?1")?; + let mut hits = Vec::with_capacity(top.len()); + for s in top { + let chunk_id = buf.ids()[s.index]; + let (source, ordinal, text) = stmt.query_row(params![chunk_id], |r| { + Ok((r.get::<_, String>(0)?, r.get::<_, u32>(1)?, r.get::<_, String>(2)?)) + })?; + hits.push(Hit { + chunk_id, + source, + ordinal, + text, + score: s.score, + }); + } + Ok(hits) + } + + /// Search straight off SQLite's rows with each blob borrowed in place: + /// the vector column only, no copy per vector, no text. Returns `(chunk + /// id, score)` for the best `k`. Measures what SQLite's row machinery + /// costs compared with the contiguous buffer. + pub fn search_borrowed( + &self, + collection: &str, + query: &[f32], + k: usize, + codec: &dyn Codec, + metric: Metric, + ) -> Result> { + self.require_collection(collection)?; + let prepared = codec.prepare_query(query)?; + let mut stmt = self + .conn + .prepare_cached("SELECT id, vector FROM chunks WHERE collection = ?1 ORDER BY id")?; + let mut rows = stmt.query(params![collection])?; + let mut all: Vec<(i64, f32)> = Vec::new(); + while let Some(row) = rows.next()? { + let blob = row.get_ref(1)?.as_blob().map_err(rusqlite::Error::from)?; + all.push((row.get(0)?, codec.score_prepared(&prepared, blob, metric)?)); + } + if metric.higher_is_better() { + all.sort_by(|a, b| b.1.total_cmp(&a.1).then(a.0.cmp(&b.0))); + } else { + all.sort_by(|a, b| a.1.total_cmp(&b.1).then(a.0.cmp(&b.0))); + } + all.truncate(k); + Ok(all) + } + + /// The original search path: every row, text included, read out of + /// SQLite and scored on every query. Kept to measure what the columnar + /// buffer saves; results are the same as [`Store::search`]. + pub fn search_by_rows( + &self, + collection: &str, + query: &[f32], + k: usize, + codec: &dyn Codec, + metric: Metric, ) -> Result> { self.require_collection(collection)?; let mut stmt = self @@ -290,6 +418,73 @@ mod tests { (store, F32Codec::new(2)) } + fn chunk(codec: &F32Codec, source: &str, ordinal: u32, v: [f32; 2]) -> NewChunk { + NewChunk { + source: source.into(), + ordinal, + text: format!("{source}#{ordinal}"), + vector: codec.encode(&v).unwrap(), + } + } + + #[test] + fn buffered_search_matches_the_row_scan_and_sees_new_chunks() { + let (mut store, codec) = store_with_collection(); + let chunks: Vec = (0..40) + .map(|i| { + let a = i as f32 * 0.15; + chunk(&codec, &format!("d{}", i % 7), i, [a.cos(), a.sin()]) + }) + .collect(); + store.insert_chunks("docs", &chunks).unwrap(); + let q = [0.6f32, 0.8]; + let rows = store.search_by_rows("docs", &q, 5, &codec, Metric::Cosine).unwrap(); + let buffered = store.search("docs", &q, 5, &codec, Metric::Cosine).unwrap(); + assert_eq!(rows, buffered); + assert_eq!(store.codes("docs").unwrap().len(), 40); + + // A write through this handle is visible to the next search. + store + .insert_chunks("docs", &[chunk(&codec, "new", 0, [0.6, 0.8])]) + .unwrap(); + let after = store.search("docs", &q, 1, &codec, Metric::Cosine).unwrap(); + assert_eq!(after[0].source, "new"); + assert_eq!(store.codes("docs").unwrap().len(), 41); + assert_eq!( + after, + store.search_by_rows("docs", &q, 1, &codec, Metric::Cosine).unwrap() + ); + } + + #[test] + fn another_connections_commit_invalidates_the_cache() { + let dir = tempfile::tempdir().unwrap(); + let path = dir.path().join("hvec.db"); + let codec = F32Codec::new(2); + let mut a = Store::open(&path).unwrap(); + a.create_collection(&NewCollection { + name: "docs", + embed_profile: "local", + embed_model: "test-model", + dimension: 2, + codec: "f32", + metric: Metric::Cosine, + codec_params: &[], + }) + .unwrap(); + a.insert_chunks("docs", &[chunk(&codec, "first", 0, [1.0, 0.0])]) + .unwrap(); + assert_eq!(a.codes("docs").unwrap().len(), 1); + let mut b = Store::open(&path).unwrap(); + b.insert_chunks("docs", &[chunk(&codec, "second", 0, [0.0, 1.0])]) + .unwrap(); + assert_eq!(a.codes("docs").unwrap().len(), 2); + let hit = a.search("docs", &[0.0, 1.0], 1, &codec, Metric::Cosine).unwrap(); + assert_eq!(hit[0].source, "second"); + a.set_mmap_size(1 << 20).unwrap(); + assert_eq!(a.load_codes("docs").unwrap().len(), 2); + } + #[test] fn insert_and_search_returns_nearest_first() { let (mut store, codec) = store_with_collection(); diff --git a/crates/hvec/src/cli.rs b/crates/hvec/src/cli.rs index 60b001c..79f72b3 100644 --- a/crates/hvec/src/cli.rs +++ b/crates/hvec/src/cli.rs @@ -161,6 +161,39 @@ pub enum BenchCommand { Run(BenchRunArgs), /// Replay an agent memory script (writes and asks) once per codec, measuring every step. Memory(MemoryArgs), + /// Time the scan itself: storage paths, codecs and thread counts, against memory bandwidth. + Scan(ScanArgs), +} + +#[derive(Debug, Args)] +pub struct ScanArgs { + /// An f32 collection; it is the ground truth and the source every codec is encoded from. + #[arg(short, long)] + pub collection: String, + /// Codecs to time on the in-memory buffer. Trained codecs are fitted on the collection first. + #[arg(long, value_delimiter = ',', default_value = "f32,int8,binary,binary-hamming,pq48")] + pub codecs: Vec, + /// Thread counts for the buffered scan. + #[arg(long, value_delimiter = ',', default_value = "1")] + pub threads: Vec, + /// Neighbours per query. + #[arg(short, long, default_value_t = 10)] + pub k: usize, + /// Stored vectors used as queries; each is excluded from its own ranking when recall is counted. + #[arg(long, default_value_t = 50)] + pub sample: usize, + /// Timed passes over the queries; the median pass is reported. + #[arg(long, default_value_t = 3)] + pub repeat: usize, + /// Size of the buffer read to measure memory bandwidth, in MiB. + #[arg(long, default_value_t = 1024)] + pub bandwidth_mib: usize, + /// Skip the slow per-query SQLite paths. + #[arg(long)] + pub skip_sqlite: bool, + /// Print the results as JSON instead of tables. + #[arg(long)] + pub json: bool, } #[derive(Debug, Args)] @@ -338,6 +371,11 @@ pub struct RescoreArgs { /// Self-queries used to calibrate the first stage's noise for the adaptive policy. #[arg(long, default_value_t = 200)] pub calibration_sample: usize, + /// Write per-query traces (first-stage ranks of the true neighbours, exact and + /// first-stage bands, margin depths on a 0.25σ grid) as JSON Lines, one line per first + /// stage, so routing policies can be replayed offline. + #[arg(long)] + pub trace: Option, /// Neighbours per query. #[arg(short, long, default_value_t = 10)] pub k: usize, diff --git a/crates/hvec/src/commands/bench.rs b/crates/hvec/src/commands/bench.rs index 99361e5..6d01231 100644 --- a/crates/hvec/src/commands/bench.rs +++ b/crates/hvec/src/commands/bench.rs @@ -24,6 +24,7 @@ pub async fn run(config: &Option, cmd: BenchCommand) -> Result<()> { BenchCommand::Rescore(args) => rescore_cmd(config, args).await, BenchCommand::Run(args) => run_cmd(config, args).await, BenchCommand::Memory(args) => crate::commands::memory::run(config, args).await, + BenchCommand::Scan(args) => crate::commands::scan::run(config, &args), } } @@ -546,6 +547,16 @@ async fn rescore_cmd(config: &Option, args: RescoreArgs) -> Result<()> // queries' exact scores never inform their own depth. let calibration = recall::self_queries(&ids, &vectors, args.calibration_sample); + // Margin grid for per-query traces: fine enough to trace the adaptive + // policy's whole recall-against-depth curve offline. + let trace_margins: Vec = (1..=24).map(|i| f64::from(i) * 0.25).collect(); + let mut trace_out = match &args.trace { + Some(path) => Some(std::io::BufWriter::new( + std::fs::File::create(path).with_context(|| format!("creating {}", path.display()))?, + )), + None => None, + }; + let mut profiles = Vec::new(); let mut cells = Vec::new(); for name in &args.codecs { @@ -566,8 +577,13 @@ async fn rescore_cmd(config: &Option, args: RescoreArgs) -> Result<()> margins: &args.margins, noise, max_depth: args.max_depth, + trace_margins: if args.trace.is_some() { &trace_margins } else { &[] }, }; - let (profile, reports) = rescore::evaluate( + let rescore::Evaluation { + profile, + reports, + traces, + } = rescore::evaluate( &ids, &vectors, &queries, @@ -600,6 +616,23 @@ async fn rescore_cmd(config: &Option, args: RescoreArgs) -> Result<()> store.insert_run(&row)?; } } + if let Some(out) = trace_out.as_mut() { + use std::io::Write as _; + let line = serde_json::json!({ + "collection": collection.name, + "first": profile.first, + "first_bytes": profile.first_bytes, + "vectors": vectors.len(), + "k": args.k, + "calibrated_noise": noise, + "trace_margins": trace_margins, + "ranks": traces.iter().map(|t| &t.ranks).collect::>(), + "band": traces.iter().map(|t| t.band).collect::>(), + "band_hat": traces.iter().map(|t| t.band_hat).collect::>(), + "margin_depths": traces.iter().map(|t| &t.margin_depths).collect::>(), + }); + writeln!(out, "{line}")?; + } profiles.push(profile); cells.extend(reports); } diff --git a/crates/hvec/src/commands/mod.rs b/crates/hvec/src/commands/mod.rs index afa8f9f..0d790be 100644 --- a/crates/hvec/src/commands/mod.rs +++ b/crates/hvec/src/commands/mod.rs @@ -6,6 +6,7 @@ pub mod list; pub mod memory; pub mod query; pub mod questions; +pub mod scan; pub mod shell; /// Shared retrieve-then-answer step used by `query` and `chat`. diff --git a/crates/hvec/src/commands/scan.rs b/crates/hvec/src/commands/scan.rs new file mode 100644 index 0000000..419865c --- /dev/null +++ b/crates/hvec/src/commands/scan.rs @@ -0,0 +1,317 @@ +//! `hvec bench scan`: how long the scan itself takes, and where the time goes. +//! +//! Three questions, one table each: +//! 1. Storage path. The same f32 search through every way bytes can reach the +//! scorer: the original row scan (every row and its text copied out of +//! SQLite on every query), SQLite rows with borrowed blobs (with SQLite's +//! memory map off and on), and the cached contiguous buffer. Also what it +//! costs to build the buffer from SQLite, or from a flat code file. +//! 2. Codec. Each codec scanned in place over its own buffer at each thread +//! count: ms per query, GB/s of codes read, recall@k against exact f32. +//! 3. The ceiling. A plain sequential read of a large buffer at the same +//! thread counts, so each scan's GB/s can be read as a fraction of what +//! the machine's memory can deliver. + +use std::path::PathBuf; +use std::time::Instant; + +use anyhow::{Result, bail}; +use hvec_core::codecs::F32Codec; +use hvec_core::{Codec, codec_by_name}; +use hvec_store::CodeBuffer; +use serde::Serialize; + +use crate::cli::ScanArgs; +use crate::context::Ctx; + +#[derive(Debug, Serialize)] +struct PathRow { + path: String, + ms_per_query: f64, + note: String, +} + +#[derive(Debug, Serialize)] +struct LoadRow { + source: String, + ms: f64, + bytes: usize, +} + +#[derive(Debug, Serialize)] +struct CodecRow { + codec: String, + bytes_per_vector: usize, + threads: usize, + ms_per_query: f64, + gb_per_s: f64, + recall_at_k: f64, +} + +#[derive(Debug, Serialize)] +struct BandwidthRow { + threads: usize, + gb_per_s: f64, +} + +#[derive(Debug, Serialize)] +struct Report { + collection: String, + vectors: usize, + dimension: usize, + queries: usize, + k: usize, + repeat: usize, + paths: Vec, + loads: Vec, + codecs: Vec, + bandwidth: Vec, +} + +/// Median over `repeat` passes of the mean ms per query. +fn time_queries(repeat: usize, queries: &[Vec], mut f: impl FnMut(&[f32]) -> Result<()>) -> Result { + // One untimed pass warms caches and builds anything lazily cached. + for q in queries.iter().take(1) { + f(q)?; + } + let mut passes = Vec::with_capacity(repeat); + for _ in 0..repeat.max(1) { + let t = Instant::now(); + for q in queries { + f(q)?; + } + passes.push(t.elapsed().as_secs_f64() * 1000.0 / queries.len() as f64); + } + passes.sort_by(f64::total_cmp); + Ok(passes[passes.len() / 2]) +} + +/// Sequential read bandwidth: sum a large buffer of u64, split over threads. +fn bandwidth(mib: usize, threads: usize, repeat: usize) -> f64 { + let words = mib * 1024 * 1024 / 8; + let data: Vec = (0..words as u64).collect(); + let per = words.div_ceil(threads.max(1)); + let mut best = f64::MAX; + for _ in 0..repeat.max(1) + 1 { + let t = Instant::now(); + let total: u64 = std::thread::scope(|s| { + let handles: Vec<_> = data + .chunks(per) + .map(|c| s.spawn(move || c.iter().fold(0u64, |a, &x| a.wrapping_add(x)))) + .collect(); + handles + .into_iter() + .map(|h| h.join().unwrap_or(0)) + .fold(0, u64::wrapping_add) + }); + std::hint::black_box(total); + best = best.min(t.elapsed().as_secs_f64()); + } + (words * 8) as f64 / best / 1e9 +} + +pub fn run(config: &Option, args: &ScanArgs) -> Result<()> { + let ctx = Ctx::load(config.as_deref())?; + let store = ctx.open_store()?; + let collection = store.require_collection(&args.collection)?; + if collection.codec != "f32" { + bail!( + "`bench scan` needs an f32 collection; `{}` is stored as {}", + collection.name, + collection.codec + ); + } + let dim = collection.dimension; + let metric = collection.metric; + let f32c = F32Codec::new(dim); + let k = args.k; + + // The collection as one buffer; this is also the exact ground truth. + let t = Instant::now(); + let exact_buf = store.load_codes(&collection.name)?; + let load_sqlite_ms = t.elapsed().as_secs_f64() * 1000.0; + let n = exact_buf.len(); + if n == 0 { + bail!("collection `{}` is empty", collection.name); + } + let vectors: Vec> = (0..n) + .map(|i| f32c.decode(exact_buf.code(i))) + .collect::>()?; + let step = n as f64 / args.sample.clamp(1, n) as f64; + let picked: Vec = (0..args.sample.clamp(1, n)) + .map(|i| ((i as f64 * step) as usize).min(n - 1)) + .collect(); + let queries: Vec> = picked.iter().map(|&i| vectors[i].clone()).collect(); + // Self-queries: each query is a stored vector, which is dropped from both + // rankings before recall is counted, as `bench recall` does. + let top_excluding = |buf: &CodeBuffer, codec: &dyn Codec, q: &[f32], skip: usize| -> Result> { + Ok(buf + .search(q, k + 1, codec, metric)? + .iter() + .map(|s| s.index) + .filter(|&i| i != skip) + .take(k) + .collect()) + }; + let truth: Vec> = queries + .iter() + .zip(&picked) + .map(|(q, &i)| top_excluding(&exact_buf, &f32c, q, i)) + .collect::>()?; + + // 1. Storage paths, f32 throughout. + let mut paths = Vec::new(); + if !args.skip_sqlite { + let ms = time_queries(args.repeat, &queries, |q| { + std::hint::black_box(store.search_by_rows(&collection.name, q, k, &f32c, metric)?); + Ok(()) + })?; + paths.push(PathRow { + path: "sqlite rows, copied".into(), + ms_per_query: ms, + note: "the original search: every row and its text copied out on every query".into(), + }); + for (label, mmap) in [ + ("sqlite rows, borrowed", 0u64), + ("sqlite rows, borrowed, mmap", 1 << 36), + ] { + store.set_mmap_size(mmap)?; + let ms = time_queries(args.repeat, &queries, |q| { + std::hint::black_box(store.search_borrowed(&collection.name, q, k, &f32c, metric)?); + Ok(()) + })?; + paths.push(PathRow { + path: label.into(), + ms_per_query: ms, + note: if mmap == 0 { + "vector column only, each blob scored in place in SQLite's page buffer".into() + } else { + "as above, with SQLite reading the file through its memory map".into() + }, + }); + } + store.set_mmap_size(0)?; + } + let ms = time_queries(args.repeat, &queries, |q| { + std::hint::black_box(store.search(&collection.name, q, k, &f32c, metric)?); + Ok(()) + })?; + paths.push(PathRow { + path: "cached buffer (Store::search)".into(), + ms_per_query: ms, + note: "codes scanned in place in one allocation; text read for the hits only".into(), + }); + + // Building the buffer: from SQLite, and from a flat code file. + let mut loads = vec![LoadRow { + source: "sqlite (borrowed blobs, one copy)".into(), + ms: load_sqlite_ms, + bytes: exact_buf.bytes().len(), + }]; + let file = std::env::temp_dir().join(format!("hvec-scan-{}-{}.codes", std::process::id(), collection.name)); + exact_buf.write_file(&file)?; + let t = Instant::now(); + let back = CodeBuffer::read_file(&file); + let _ = std::fs::remove_file(&file); + let back = back?; + loads.push(LoadRow { + source: "flat code file (one read)".into(), + ms: t.elapsed().as_secs_f64() * 1000.0, + bytes: back.bytes().len(), + }); + drop(back); + + // 2. Codecs, each scanned in place over its own buffer. + let ids = exact_buf.ids().to_vec(); + let sources = exact_buf.sources().to_vec(); + let mut codecs = Vec::new(); + for name in &args.codecs { + let mut codec: Box = codec_by_name(name, dim)?; + if codec.needs_fit() { + codec.fit(&vectors)?; + } + let buf = CodeBuffer::encode(codec.as_ref(), &ids, &sources, &vectors)?; + let mut hits = 0usize; + for ((q, t), &i) in queries.iter().zip(&truth).zip(&picked) { + let got = top_excluding(&buf, codec.as_ref(), q, i)?; + hits += got.iter().filter(|g| t.contains(g)).count(); + } + let recall = hits as f64 / (queries.len() * k) as f64; + for &threads in &args.threads { + let ms = time_queries(args.repeat, &queries, |q| { + std::hint::black_box(buf.search_threads(q, k, codec.as_ref(), metric, threads)?); + Ok(()) + })?; + codecs.push(CodecRow { + codec: codec.name().to_owned(), + bytes_per_vector: codec.encoded_size(), + threads, + ms_per_query: ms, + gb_per_s: buf.bytes().len() as f64 / (ms / 1000.0) / 1e9, + recall_at_k: recall, + }); + } + } + + // 3. The memory-bandwidth ceiling. + let bandwidth = args + .threads + .iter() + .map(|&t| BandwidthRow { + threads: t, + gb_per_s: bandwidth(args.bandwidth_mib, t, args.repeat), + }) + .collect(); + + let report = Report { + collection: collection.name.clone(), + vectors: n, + dimension: dim, + queries: queries.len(), + k, + repeat: args.repeat, + paths, + loads, + codecs, + bandwidth, + }; + if args.json { + println!("{}", serde_json::to_string_pretty(&report)?); + return Ok(()); + } + println!( + "collection `{}` · {} vectors · {} dims · {} queries · k={} · median of {} passes", + report.collection, report.vectors, report.dimension, report.queries, k, report.repeat + ); + println!("\nstorage path (f32) ms/query"); + for p in &report.paths { + println!(" {:<32} {:>8.2} {}", p.path, p.ms_per_query, p.note); + } + println!("\nbuilding the buffer ms MB"); + for l in &report.loads { + println!(" {:<32} {:>8.1} {:>8.1}", l.source, l.ms, l.bytes as f64 / 1e6); + } + println!("\ncodec, in place bytes threads ms/query GB/s % of bandwidth recall@{k}"); + for c in &report.codecs { + let bw = report + .bandwidth + .iter() + .find(|b| b.threads == c.threads) + .map_or(f64::NAN, |b| b.gb_per_s); + println!( + " {:<18} {:>6} {:>8} {:>9.2} {:>8.2} {:>14.1}% {:>9.4}", + c.codec, + c.bytes_per_vector, + c.threads, + c.ms_per_query, + c.gb_per_s, + 100.0 * c.gb_per_s / bw, + c.recall_at_k + ); + } + println!("\nmemory bandwidth (sequential u64 read of {} MiB)", args.bandwidth_mib); + for b in &report.bandwidth { + println!(" {:>2} thread(s): {:>6.1} GB/s", b.threads, b.gb_per_s); + } + Ok(()) +} diff --git a/crates/hvec/tests/pipeline.rs b/crates/hvec/tests/pipeline.rs index 0393363..6dce31c 100644 --- a/crates/hvec/tests/pipeline.rs +++ b/crates/hvec/tests/pipeline.rs @@ -1193,3 +1193,57 @@ fn memory_script_replays_per_codec_and_accounts_bytes() { 18 ); } + +#[test] +fn bench_scan_times_paths_and_codecs_and_keeps_f32_exact() { + let env = Env::new(); + for i in 0..6 { + env.write_doc( + &format!("extra{i}.md"), + &format!("Topic {i}: notes about subject number {i} and its details, item {i}."), + ); + } + env.write_doc("refunds.md", REFUNDS); + env.write_doc("shipping.md", SHIPPING); + let dir = env.dir.path().to_str().unwrap().to_owned(); + assert_ok(&env.run(&["ingest", &dir, "--collection", "docs"]), "ingest dir"); + + let out = env.run(&[ + "bench", + "scan", + "--collection", + "docs", + "--codecs", + "f32,binary,binary-hamming", + "--threads", + "1,2", + "-k", + "3", + "--sample", + "4", + "--repeat", + "1", + "--bandwidth-mib", + "8", + "--json", + ]); + assert_ok(&out, "bench scan"); + let r: serde_json::Value = serde_json::from_str(&stdout(&out)).expect("json report"); + assert_eq!(r["vectors"], 8); + assert_eq!(r["queries"], 4); + let paths: Vec<&str> = r["paths"] + .as_array() + .unwrap() + .iter() + .map(|p| p["path"].as_str().unwrap()) + .collect(); + assert_eq!(paths.len(), 4, "{paths:?}"); + assert_eq!(r["loads"].as_array().unwrap().len(), 2); + let codecs = r["codecs"].as_array().unwrap(); + assert_eq!(codecs.len(), 6, "three codecs at two thread counts"); + for c in codecs.iter().filter(|c| c["codec"] == "f32") { + assert_eq!(c["recall_at_k"], 1.0, "{c}"); + } + assert!(codecs.iter().all(|c| c["ms_per_query"].as_f64().unwrap() >= 0.0)); + assert_eq!(r["bandwidth"].as_array().unwrap().len(), 2); +} diff --git a/docs/_experiments/hybrid-adaptive-matrix.md b/docs/_experiments/hybrid-adaptive-matrix.md new file mode 100644 index 0000000..a2d75c7 --- /dev/null +++ b/docs/_experiments/hybrid-adaptive-matrix.md @@ -0,0 +1,45 @@ +--- +title: "The hybrid × adaptive matrix" +date: 2026-11-04 21:00:00 +0000 +result: "PCA plus variance-weighted bit allocation beats product quantization at equal bytes on average (0.768 vs 0.747 recall@10 at 48 bytes), all of it on MiniLM. Adaptive re-score depth gains half a point at equal work, up to 4 points for the worst queries." +references: [li2024quantmoe, huang1963block, max1960quantizing, jegou2011pq, thakur2021beir, xiao2023bge, reimers2019sbert] +--- + +*Full write-up, raw results and a one-command reproduction are in the repository:* +[experiments/2026-11-hybrid-adaptive-matrix](https://github.com/SwineCoder101/hvec/tree/main/experiments/2026-11-hybrid-adaptive-matrix). +Follow-up to [two-stage retrieval]({{ "/experiments/two-stage-rescoring/" | relative_url }}). + +

Animated bar chart of a 48-byte budget spread over 384 principal components of SciFact's bge-small embeddings: uniform binary, variance-based allocation, and query-weighted allocation.

+ +**Two kinds of adaptive compression.** *Adaptive bits* spend a scan code's bytes unevenly, the +idea behind structure-aware mixed-precision quantization of mixture-of-experts models [1]. +`mixed` gives each dimension 0 to 8 bits by reverse water-filling on its variance [2], with a +Gaussian-optimal quantizer step [3]. `-pca` rotates into principal components first; `-w` weights +each component by the query's energy in it. *Adaptive depth* re-scores every candidate within λσ +of the scan's own 10th score, with σ calibrated on self-queries. + +**Setup.** Ten collections (five BEIR corpora [5] × bge-small [6] and MiniLM [7]); eleven scan +codecs at 24, 48 and 96 bytes, uniform (`binary`, `binary-centred`, `pq` [4]) and adaptive +(`mixed`); no re-score, fixed depths 20/50/100, or margins 1σ/2σ/3σ; int8 or f32 second stage. + +**Result.** + +| bytes | best uniform scan, recall@10 | `mixed-pca-w`, recall@10 | bge-small | MiniLM | +|---:|---|---:|---|---| +| 24 | `pq24` 0.606 | **0.635** | 0.587 → 0.592 | 0.624 → 0.677 | +| 48 | `pq48` 0.747 | **0.768** | 0.724 → 0.723 | 0.769 → 0.813 | +| 96 | `pq96` 0.870 | **0.886** | 0.855 → 0.855 | 0.885 → 0.916 | + +Without the rotation, `mixed48` (0.696) is no better than `binary-centred` (0.698). Measured +exactly at equal mean depth, adaptive depth adds 0.3–0.6 points of recall@10 over a fixed depth +(ahead in 156 of 180 comparisons) and up to 4 points of worst-decile recall, at a 90th-percentile +depth 1.3–2.1× the mean. *Correction:* the first version of the write-up said 1.5 points, from +interpolating between too few fixed depths. + +**Reading.** PCA plus bit allocation never trails PQ by more than a point at equal bytes and +leads by 3–5 points on MiniLM. On bge-small, where 60% of a vector's energy is a shared mean +direction, the two tie; PQ's codebooks probably absorb that component more cheaply. The cost: +`mixed48-pca-w` is the slowest scan measured, 2.2× f32 per vector ([zero-copy scan]({{ "/experiments/zero-copy-scan/" | relative_url }})). +The full matrix and a benefits-and-drawbacks table for every combination are in the write-up. + +{% include references.html %} diff --git a/docs/_experiments/routing-oracle.md b/docs/_experiments/routing-oracle.md new file mode 100644 index 0000000..081a4e2 --- /dev/null +++ b/docs/_experiments/routing-oracle.md @@ -0,0 +1,43 @@ +--- +title: "Routing: choosing the compression per query or per collection" +date: 2026-11-04 22:00:00 +0000 +result: "Perfect per-query routing would gain 5.5–7 points of recall@10 over a fixed re-score depth, but the scan's own scores reveal almost none of it: no router built on them captures more than 6%. Per collection, the noise-to-spacing curve picks the right codec but not the right depth." +references: [li2024quantmoe, thakur2021beir, xiao2023bge, reimers2019sbert] +--- + +*Full write-up, raw results and a one-command reproduction are in the repository:* +[experiments/2026-11-routing-oracle](https://github.com/SwineCoder101/hvec/tree/main/experiments/2026-11-routing-oracle). +Follow-up to [the hybrid × adaptive matrix]({{ "/experiments/hybrid-adaptive-matrix/" | relative_url }}). + +

Animated schematic of adaptive re-scoring depth: a crowded query and a well-separated query share one noise margin but get very different numbers of candidates.

+ +**Hypothesis.** Routing each query to the compression that suits it, the way a mixture-of-experts +gate routes tokens [1], has real headroom, because queries need very different re-score depths. +But a router sees only the scan's own noisy scores, so it will capture little of that headroom. + +**Setup.** `bench rescore --trace` records, for 38,124 queries over six scan codecs and ten +collections [2, 3, 4], where the scan ranks each true neighbour and the band visible in the +scan's own scores. With an exact second stage that is enough to replay any depth policy offline: +fixed depth, the margin policy, a ratio router (depth from σ ÷ the visible band), and a hindsight +oracle solved exactly for each budget. + +**Result.** Confirmed for per-query routing; half confirmed per collection. + +| mean depth | fixed | margin | ratio router | oracle | +|---:|---:|---:|---:|---:| +| 20 | 0.832 | −0.000 | −0.020 | +0.071 | +| 30 | 0.875 | +0.003 | −0.008 | +0.069 | +| 50 | 0.900 | +0.004 | −0.001 | +0.067 | +| 100 | 0.939 | +0.003 | +0.000 | +0.055 | + +The depth a query needs varies 6.3× between its 10th and 90th percentile, but the visible band +tracks the true band only at Spearman 0.52–0.70. Per collection, predicting the cheapest +(codec, depth) from the noise-to-spacing curve picks the right scan size in 10 of 10 collections, +yet reaches anywhere from 0.86 to 1.00 when aiming for 95%. + +**Reading.** The oracle's headroom is mostly chance: which neighbour the code's noise happened to +push down, which the same noisy scores cannot reveal. Keep the cheap margin policy for its +worst-query gain; choose the codec per collection from the curve; measure, don't predict, the +depth. + +{% include references.html %} diff --git a/docs/_experiments/two-stage-rescoring.md b/docs/_experiments/two-stage-rescoring.md new file mode 100644 index 0000000..14f4172 --- /dev/null +++ b/docs/_experiments/two-stage-rescoring.md @@ -0,0 +1,46 @@ +--- +title: "Two-stage retrieval: how deep must the re-score go?" +date: 2026-11-04 20:00:00 +0000 +result: "The depth an f32 re-score needs follows the same noise-to-spacing ratio that predicts codec loss (Spearman ≈ 0.93). At 48 bytes, 50 f32 candidates recover 0.95 (binary) to 0.99 (pq48) of the exact top 10 and nearly all of the nDCG loss." +references: [thakur2021beir, xiao2023bge, reimers2019sbert, jegou2011pq, charikar2002simhash] +--- + +*Full write-up, raw results and a one-command reproduction are in the repository:* +[experiments/2026-11-two-stage-rescoring](https://github.com/SwineCoder101/hvec/tree/main/experiments/2026-11-two-stage-rescoring). +Follow-up to [neighbour spacing]({{ "/experiments/neighbour-spacing/" | relative_url }}). + +

Animated schematic of two-stage retrieval: a small code shortlists candidates, an exact f32 re-score lifts true neighbours into the top 10, and one neighbour ranked outside the shortlist is missed.

+ +**Hypothesis.** A codec with noise σ displaces a true neighbour past about (σ ÷ band) × k +competitors, so the candidate depth an exact re-score needs grows with the noise-to-spacing +ratio, roughly in proportion to it. + +**Setup.** The ten collections of the spacing experiment: five BEIR-format corpora [1] × bge-small +[2] and MiniLM [3]. Scan codecs `binary`, `binary-centred` [5], `pq96`, `pq48`, `pq24` [4]; +second stage `int8` or `f32`, read for candidates only; depths 10 to 1,000. With an exact second +stage, the depth needed for a target recall is a quantile of where the scan ranks the true +neighbours, so it is measured exactly. + +**Result.** Confirmed, with one correction. The depth tracks the ratio with ρ ≈ 0.93 at every +target. It is proportional at 90% recall but grows faster than proportional for higher targets, +because the last neighbours sit in the tail of the code's error: + +| target recall@10 | depth / k | Spearman ρ | held-out corpus error | +|---|---|---:|---:| +| 90% | 3.07 × r^1.06 | 0.930 | ×1.25 | +| 95% | 4.76 × r^1.29 | 0.934 | ×1.31 | +| 99% | 11.22 × r^1.74 | 0.925 | ×1.41 | + +| first stage | depth 10 | 20 | 50 | 100 | nDCG loss at depth 10 → 50 | +|---|---:|---:|---:|---:|---| +| binary, f32 re-score | 0.669 | 0.841 | 0.948 | 0.981 | 0.0138 → 0.0003 | +| pq48, f32 re-score | 0.747 | 0.913 | 0.986 | 0.998 | 0.0078 → 0.0002 | +| pq48, int8 re-score | 0.747 | 0.910 | 0.979 | 0.989 | 0.0078 → 0.0002 | +| pq96, f32 re-score | 0.870 | 0.986 | 1.000 | 1.000 | 0.0005 → −0.0001 | + +**Reading.** Re-scoring 50 candidates costs 75 KB per query on top of the scan, 3–4% of the +bytes an f32 scan reads. An int8 second stage matches f32 up to a depth of about 30, then stops +at int8's own 0.990. Caveats: brute force only; the f32 copy must still be stored; one fixed +depth for every query. + +{% include references.html %} diff --git a/docs/_experiments/zero-copy-scan.md b/docs/_experiments/zero-copy-scan.md new file mode 100644 index 0000000..8f0d222 --- /dev/null +++ b/docs/_experiments/zero-copy-scan.md @@ -0,0 +1,49 @@ +--- +title: "Zero-copy scan: which codecs are actually faster?" +date: 2026-11-04 23:00:00 +0000 +result: "Copying was most of the original search's cost: a contiguous in-place scan is 5.1× faster. Fewer bytes do not make a scan faster, except with popcount: binary-hamming plus an f32 re-score reaches 95% recall@10 on FiQA about 19× faster than exact f32." +references: [charikar2002simhash, jegou2011pq, thakur2021beir, maia2018fiqa, xiao2023bge] +--- + +*Full write-up, raw results and a one-command reproduction are in the repository:* +[experiments/2026-11-zero-copy-scan](https://github.com/SwineCoder101/hvec/tree/main/experiments/2026-11-zero-copy-scan). + +**Hypothesis.** Most of the original search's time is copying, not scoring: every query read +every row, blob and text out of SQLite and allocated a vector per row. And once copying is gone, +asymmetric codecs (an f32 query against compressed codes) will still be no faster than f32, +because their scoring, not their bytes, sets the time. + +**Setup.** `hvec bench scan` on SciFact (8,036 vectors) and FiQA [4] (71,274), both from BEIR [3], bge-small [5], +384 dims; codes scanned in place in one contiguous buffer, borrowed by the codec as `&[u8]`; 1 to +10 threads; a sequential-read bandwidth ceiling for comparison. Apple M4 Pro, idle. +`binary-hamming` stores `binary`'s sign bits [1] but binarises the query too and scores by XOR +and popcount. + +**Result.** Both confirmed. + +| f32 search path, FiQA | ms / query | +|---|---:| +| SQLite rows copied (the original) | 74.7 | +| SQLite rows, blobs borrowed | 54.5 | +| … with SQLite's memory map | 27.4 | +| contiguous buffer, in place | **14.7** | + +| codec, one thread, FiQA | bytes | ns / vector | share of bandwidth | +|---|---:|---:|---:| +| f32 | 1,536 | 207 | 9.5% | +| int8 | 392 | 261 | 1.9% | +| binary | 48 | 246 | 0.3% | +| pq48 [2] | 48 | 203 | 0.3% | +| mixed48-pca-w | 48 | 462 | 0.1% | +| binary-hamming | 48 | **10** | 6.1% | + +A `binary-hamming` scan alone keeps only 0.40 of FiQA's top 10, but re-scoring 369 candidates in +f32 reaches 95%, for about 0.79 ms against 14.8 ms for exact f32 on one thread. That is roughly +19× faster, with nDCG within 0.0003 of f32 at 500 candidates. + +**Reading.** Compression buys memory everywhere, and time only with symmetric scoring. Caveats: +brute force; the hybrid's re-score cost is computed from measured per-vector costs, assuming the +f32 vectors are in memory; no explicit SIMD; one machine. A memory-mapped code file is left out +because the workspace forbids `unsafe`. + +{% include references.html %} diff --git a/docs/_posts/2026-11-05-hybrid-adaptive-and-fast.md b/docs/_posts/2026-11-05-hybrid-adaptive-and-fast.md new file mode 100644 index 0000000..e24fdc9 --- /dev/null +++ b/docs/_posts/2026-11-05-hybrid-adaptive-and-fast.md @@ -0,0 +1,67 @@ +--- +title: "Hybrid, adaptive, and finally fast" +date: 2026-11-05 10:00:00 +0000 +references: [li2024quantmoe, charikar2002simhash, jegou2011pq] +--- + +[The last post]({{ "/blog/2026/11/04/does-the-kind-of-text-matter/" | relative_url }}) found the +number that sets what a codec costs: its score noise divided by how tightly the corpus packs a +query's neighbours. Four experiments followed from it. Together they answer the three questions +people actually ask about vector compression: how do I get the lost neighbours back, can the +compression adapt to the data, and is any of it faster? + +## Getting the neighbours back: hybrid retrieval + +Scan with a small code, shortlist its best N candidates, and re-score only those with the full +vectors. + +

Animated schematic of two-stage retrieval.

+ +The same noise-to-spacing ratio that predicts the loss also predicts how deep the shortlist must +be, to within about a third. At 48 bytes per vector, 50 candidates recover 95–99% of the exact +top 10 and essentially all of the loss in nDCG +([two-stage retrieval]({{ "/experiments/two-stage-rescoring/" | relative_url }})). + +## Adaptive compression + +Adaptive compression can mean spending bits unevenly, or spending work unevenly. + +**Uneven bits.** Mixed-precision quantization of mixture-of-experts models gives more bits to the +blocks and experts that matter more [1]. For embeddings, "matters more" means a principal +component with more variance, or with more of the query's energy: + +

Animated bar chart of a 48-byte bit allocation over 384 principal components.

+ +Rotating into principal components and allocating bits this way beats product quantization [3] at +24, 48 and 96 bytes on average, by 1.6 to 2.9 points of recall@10. The gain is all on one of the two +embedding models; on the other they tie +([the hybrid × adaptive matrix]({{ "/experiments/hybrid-adaptive-matrix/" | relative_url }})). + +**Uneven work.** Re-score more candidates for crowded queries and fewer for well-separated ones: + +

Animated schematic of adaptive re-scoring depth.

+ +This works, but only a little: half a point of recall at the same average work, and up to four +points for the worst queries. I first reported 1.5 points; that came from comparing against too +few fixed depths, and the write-up now says so. A hindsight oracle shows 6 points of headroom, +but the scan's own scores reveal almost none of it, and no router built on them captured more than +6% ([routing]({{ "/experiments/routing-oracle/" | relative_url }})). Routing is worth doing per +collection, where the noise-to-spacing curve picks the right codec, but not per query. + +## Is any of it faster? + +Until now hvec claimed nothing about speed. Zero-copy plumbing changed that: the codes now sit in +one contiguous buffer and are scored in place, with no copy or allocation per vector. That alone +made search 5× faster. Then the uncomfortable result. With copying out of the way, every +asymmetric codec (an f32 query against compressed codes) costs about what f32 costs per vector, +because its scoring, not its bytes, sets the time. A 48-byte binary scan reads 32× fewer bytes +and takes longer than f32. + +The exception is symmetric scoring: binarise the query too and compare by XOR and popcount [2]. That +costs 10 ns per vector against 207 for f32. On its own it keeps only 40% of the top 10, but with +an f32 re-score of a few hundred candidates it reaches 95% recall about 19× faster than an exact +f32 scan ([zero-copy scan]({{ "/experiments/zero-copy-scan/" | relative_url }})). + +So compression buys memory everywhere, and time only when the scoring is as cheap as the read. + +{% include references.html %} diff --git a/docs/assets/animations/adaptive-bits.svg b/docs/assets/animations/adaptive-bits.svg new file mode 100644 index 0000000..41b415b --- /dev/null +++ b/docs/assets/animations/adaptive-bits.svg @@ -0,0 +1,215 @@ + +Adaptive bits: mixed-precision allocation at 48 bytes +Animated bar chart of how a 48-byte budget is spread over 384 principal components of SciFact's bge-small embeddings. Uniform binary gives every component 1 bit (recall@10 0.611). Variance-based allocation concentrates bits on the leading components and gives 97 components none (0.694); query-weighted allocation goes further, 158 components with none (0.703). + + +Adaptive bits: the same 48 bytes, spent where they matter +0.001 +0.01 +0.1 +1 + +variance per principal component, relative to the first (log scale, clipped at 0.001) +0 +8 +16 +24 +32 +40 +bits per group of 8 components (48 bytes = 384 bits in all) + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +component 1component 384 +Uniform: 1 bit on every dimension (binary, 48 B) +mixed48-pca: rotate to principal components, spend bits where the variance is +mixed48-pca-w: weight variance by how much energy queries put there +recall@10 without re-scoring: 0.611 · components with 0 bits: 0 of 384 +recall@10 without re-scoring: 0.694 · components with 0 bits: 97 of 384 +recall@10 without re-scoring: 0.703 · components with 0 bits: 158 of 384 +bits return to the last 44 components: little variance,but 28% of the corpus mean, which every query points along +Real data: SciFact, bge-small-en-v1.5, 8,036 chunks, 300 claims. pq48 at the same 48 bytes reaches 0.711.Mixed precision by importance is the idea behind structure-aware MoE quantization (QuantMoE-Bench). + + diff --git a/docs/assets/animations/adaptive-depth.svg b/docs/assets/animations/adaptive-depth.svg new file mode 100644 index 0000000..2d3f974 --- /dev/null +++ b/docs/assets/animations/adaptive-depth.svg @@ -0,0 +1,264 @@ + +Adaptive depth: a per-query margin instead of a fixed shortlist +Animated schematic of adaptive re-scoring depth. Two queries share one noise-based margin below their 10th code score. The crowded query has 59 candidates inside the margin, the well-separated one 18; a fixed shortlist of 30 would under-fetch for the first and over-fetch for the second. + + +Adaptive depth: re-score every candidate within λσ of the 10th score +The code's score noise σ is calibrated once per collection, on self-queries. +The margin λσ is the same for every query; how many candidates fall inside it is not. +A fixed shortlist (dashed) over-fetches for one query and under-fetches for the other. +Crowded query: many candidates score close to the 10th + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +10th + +fixed N = 30 +re-scores 59 candidates +Well-separated query: the 10th stands clear of the rest + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +10th + +fixed N = 30 +re-scores 18 candidates +the code's top 10re-scored by marginnot re-scored +Schematic: synthetic code scores, one σ for both queries. Measured results: experiments/2026-11-hybrid-adaptive-matrix. + + diff --git a/docs/assets/animations/hybrid-rescoring.svg b/docs/assets/animations/hybrid-rescoring.svg new file mode 100644 index 0000000..ea51ee3 --- /dev/null +++ b/docs/assets/animations/hybrid-rescoring.svg @@ -0,0 +1,197 @@ + +Hybrid quantization: two-stage re-scoring +Animated schematic of two-stage retrieval. 44 vectors are scored from small codes; the code's top 10 holds 6 of the 10 true neighbours. The code's top 30 are re-scored with exact f32 vectors, which lifts 9 true neighbours into the top 10; one, ranked 31 by the code, falls outside the shortlist and is missed. + + +Hybrid quantization: scan with a small code, re-score a shortlist exactly + +shortlist: the code's top 30 (f32 fetched for these only) +top 10 → + +higher score → +each dot is one stored vector, placed by its score against the query + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +missed: the code ranked it #31, outside the shortlist +1 · Scan. Every vector is scored from its 48-byte code, in memory. +2 · Cut at the code's top 10: it keeps 6 of the 10 true neighbours. +3 · Instead, shortlist the code's top 30 and fetch their f32 vectors. +4 · Re-score the shortlist exactly. True neighbours rise into the top 10. +Result: 9 of 10 true neighbours. A deeper shortlist catches the last one. +true top-10 neighbour (exact f32)other vectortrue neighbour the hybrid missed +Schematic: 44 synthetic vectors. Measured over ten collections, re-scoring 50 f32 candidates recovers0.95 (binary) to 0.99 (pq48) of the top 10, for 3–4% of the bytes an f32 scan reads. + + diff --git a/docs/assets/animations/make_animations.py b/docs/assets/animations/make_animations.py new file mode 100644 index 0000000..1b5ed42 --- /dev/null +++ b/docs/assets/animations/make_animations.py @@ -0,0 +1,384 @@ +"""Generate the README's animated SVGs for hybrid and adaptive quantization. + + python3 -I docs/assets/animations/make_animations.py + +Writes hybrid-rescoring.svg, adaptive-depth.svg and adaptive-bits.svg next to this file. The +first two are schematic (synthetic scores, fixed seed). adaptive-bits.svg uses real data: +SciFact's bge-small-en-v1.5 vectors, their PCA spectrum, and the 48-byte bit allocations that +`mixed48-pca` and `mixed48-pca-w` make, stored in scifact-bge-small-alloc48.json. + +The animations are CSS keyframes, so GitHub renders them through . Under +prefers-reduced-motion they stop on their final, fully labelled frame. Colours are the dataviz +reference palette (blue and orange slots, validated all-pairs in both modes), with light and dark +steps selected by prefers-color-scheme. +""" + +import json +import random +from pathlib import Path + +HERE = Path(__file__).resolve().parent +FONT = 'ui-sans-serif, system-ui, -apple-system, "Segoe UI", Helvetica, Arial, sans-serif' + +BASE_CSS = f""" + svg {{ --surface:#fcfcfb; --ink:#0b0b0b; --ink2:#52514e; --muted:#898781; --grid:#e1e0d9; + --axis:#c3c2b7; --s1:#2a78d6; --s2:#eb6834; --other:#898781; + --band:rgba(42,120,214,0.10); --band2:rgba(42,120,214,0.20); }} + @media (prefers-color-scheme: dark) {{ + svg {{ --surface:#1a1a19; --ink:#f0efec; --ink2:#c3c2b7; --grid:#2c2c2a; --axis:#383835; + --s1:#3987e5; --s2:#d95926; --band:rgba(57,135,229,0.16); --band2:rgba(57,135,229,0.28); }} + }} + text {{ font-family: {FONT}; fill: var(--ink); }} + .bg {{ fill: var(--surface); stroke: var(--grid); }} + .h {{ font-size: 15px; font-weight: 600; }} + .cap {{ font-size: 13px; fill: var(--ink2); }} + .small {{ font-size: 11px; fill: var(--muted); }} + .lbl {{ font-size: 12px; fill: var(--ink2); }} + .num {{ font-size: 12px; fill: var(--ink); font-variant-numeric: tabular-nums; }} + .axis {{ stroke: var(--axis); stroke-width: 1; }} + .grid {{ stroke: var(--grid); stroke-width: 1; }} + .true {{ fill: var(--s1); stroke: var(--surface); stroke-width: 2; }} + .other {{ fill: var(--other); fill-opacity: .55; stroke: var(--surface); stroke-width: 2; }} + .lost {{ fill: var(--s2); stroke: var(--surface); stroke-width: 2; }} + .cut {{ stroke: var(--ink); stroke-width: 1.5; stroke-dasharray: 4 3; }} + .fixed {{ stroke: var(--s2); stroke-width: 2; stroke-dasharray: 5 3; }} + .band {{ fill: var(--band); }} + .band2 {{ fill: var(--band2); }} +""" +REDUCED = "@media (prefers-reduced-motion: reduce) { * { animation: none !important; } }" + + +def svg(w, h, title, desc, css, body): + return f""" +{title} +{desc} + + +{body} + +""" + + +def phase(name, dur, stops): + """A keyframe rule from [(percent, css)] stops.""" + frames = " ".join(f"{p}% {{ {css} }}" for p, css in stops) + return f"@keyframes {name} {{ {frames} }}\n" + + +def captions(prefix, dur, items, y, x=24, cls="cap"): + """Cross-fading captions; the last is the reduced-motion resting frame.""" + css, body = "", "" + n = len(items) + for i, (start, end, text) in enumerate(items): + last = i == n - 1 + name = f"{prefix}{i}" + a, b = start, end + stops = [(0, "opacity:0"), (max(a - 0.01, 0), "opacity:0"), (min(a + 2, b), "opacity:1"), + (max(b - 2, a), "opacity:1"), (b, "opacity:0"), (100, "opacity:0")] + if a == 0: + stops = [(0, "opacity:1")] + stops[3:] + css += phase(name, dur, stops) + css += f".{name} {{ opacity:{1 if last else 0}; animation: {name} {dur}s infinite; }}\n" + body += f'{text}\n' + return css, body + + +# --------------------------------------------------------------------------- +# 1. Hybrid re-scoring +# --------------------------------------------------------------------------- + +def hybrid(): + rng = random.Random(11) + n, k, depth = 44, 10, 30 + # Find a draw that shows the story: the code's top 10 keeps 6 true neighbours, the top 30 + # keeps 9, one is ranked beyond the shortlist. + while True: + exact = sorted((rng.gauss(0, 1) for _ in range(n)), reverse=True) + approx = [e + rng.gauss(0, 0.55) for e in exact] + order = sorted(range(n), key=lambda i: -approx[i]) + rank = {i: r for r, i in enumerate(order)} + true = set(range(k)) + in10 = sum(1 for i in true if rank[i] < k) + in30 = sum(1 for i in true if rank[i] < depth) + if in10 == 6 and in30 == 9: + break + lost = next(i for i in true if rank[i] >= depth) + short = set(order[:depth]) + rescored_order = sorted(short, key=lambda i: -exact[i]) + final10 = set(rescored_order[:k]) + assert len(final10 & true) == 9 + + W, H = 820, 370 + x0, x1 = 50, 790 + lo = min(min(exact), min(approx)) - 0.1 + hi = max(max(exact), max(approx)) + 0.1 + X = lambda s: x0 + (s - lo) / (hi - lo) * (x1 - x0) + lanes = [150, 166, 182, 198, 214, 230] + dur = 16 + css = "" + body = 'Hybrid quantization: scan with a small code, re-score a shortlist exactly\n' + + # The code's 10th and the shortlist edge, both on approx scores; the final 10th on exact. + a_sorted = sorted(approx, reverse=True) + cut10 = (a_sorted[k - 1] + a_sorted[k]) / 2 + cut30 = (a_sorted[depth - 1] + a_sorted[depth]) / 2 + e_short = sorted((exact[i] for i in short), reverse=True) + final_cut = (e_short[k - 1] + e_short[k]) / 2 + + # Shortlist band (phase 2 on) and top-10 zone. + css += phase("sl", dur, [(0, "opacity:0"), (28, "opacity:0"), (32, "opacity:1"), (94, "opacity:1"), (100, "opacity:0")]) + css += ".sl { opacity:1; animation: sl %ds infinite; }\n" % dur + body += f'\n' + body += f'shortlist: the code\'s top {depth} (f32 fetched for these only)\n' + + # Top-10 cut: appears at the code's 10th, then moves to the exact 10th after re-scoring. + shift = X(final_cut) - X(cut10) + css += phase("cut", dur, [(0, "opacity:0;transform:translateX(0)"), (12, "opacity:0;transform:translateX(0)"), + (16, "opacity:1;transform:translateX(0)"), (50, "opacity:1;transform:translateX(0)"), + (62, f"opacity:1;transform:translateX({shift:.1f}px)"), + (94, f"opacity:1;transform:translateX({shift:.1f}px)"), + (100, f"opacity:0;transform:translateX({shift:.1f}px)")]) + css += f".cutg {{ transform: translateX({shift:.1f}px); animation: cut {dur}s infinite; }}\n" + body += (f'' + f'top 10 →\n') + + # Axis. + body += f'\n' + body += f'higher score →\n' + body += f'each dot is one stored vector, placed by its score against the query\n' + + # Dots. Shortlisted ones slide from the code's score to the exact score. + for i in range(n): + y = lanes[i % len(lanes)] + cls = "true" if i in true else "other" + dx = X(exact[i]) - X(approx[i]) if i in short else 0.0 + name = f"d{i}" + stops = [(0, "opacity:0;transform:translateX(0)"), (4, "opacity:1;transform:translateX(0)"), + (50, "opacity:1;transform:translateX(0)"), (62, f"opacity:1;transform:translateX({dx:.1f}px)"), + (94, f"opacity:1;transform:translateX({dx:.1f}px)"), (100, f"opacity:0;transform:translateX({dx:.1f}px)")] + if i not in short and i != lost: + stops = [(0, "opacity:0"), (4, "opacity:1"), (50, "opacity:1"), (60, "opacity:.35"), + (94, "opacity:.35"), (100, "opacity:0")] + rest = "opacity:.35" + else: + rest = f"transform:translateX({dx:.1f}px)" + css += phase(name, dur, stops) + css += f".{name} {{ {rest}; animation: {name} {dur}s infinite; }}\n" + if i == lost: + css += phase("lostc", dur, [(0, "fill:var(--s1)"), (64, "fill:var(--s1)"), (68, "fill:var(--s2)"), + (100, "fill:var(--s2)")]) + body += f'\n' + lost_xy = (X(approx[i]), y) + else: + body += f'\n' + + # Label for the missed neighbour, shown in the last phase. + css += phase("ml", dur, [(0, "opacity:0"), (66, "opacity:0"), (70, "opacity:1"), (94, "opacity:1"), (100, "opacity:0")]) + css += f".ml {{ opacity:1; animation: ml {dur}s infinite; }}\n" + lx, ly = lost_xy + body += (f'' + f'missed: the code ranked it #{rank[lost] + 1}, outside the shortlist\n') + + # Phase counters. + c, b = captions("hc", dur, [ + (0, 14, "1 · Scan. Every vector is scored from its 48-byte code, in memory."), + (14, 30, f"2 · Cut at the code's top 10: it keeps {in10} of the 10 true neighbours."), + (30, 50, f"3 · Instead, shortlist the code's top {depth} and fetch their f32 vectors."), + (50, 66, "4 · Re-score the shortlist exactly. True neighbours rise into the top 10."), + (66, 100, f"Result: {len(final10 & true)} of 10 true neighbours. A deeper shortlist catches the last one."), + ], y=64) + css += c + body += b + + # Legend (two series plus the missed state; text in ink, colour on the mark). + lg = 312 + body += (f'true top-10 neighbour (exact f32)' + f'other vector' + f'true neighbour the hybrid missed\n') + body += (f'Schematic: {n} synthetic vectors. Measured over ten collections, re-scoring 50 f32 candidates recovers' + f'0.95 (binary) to 0.99 (pq48) of the top 10, for 3–4% of the bytes an f32 scan reads.\n') + + desc = (f"Animated schematic of two-stage retrieval. {n} vectors are scored from small codes; the code's top 10 holds " + f"{in10} of the 10 true neighbours. The code's top {depth} are re-scored with exact f32 vectors, which lifts " + f"{len(final10 & true)} true neighbours into the top 10; one, ranked {rank[lost] + 1} by the code, falls outside " + "the shortlist and is missed.") + return svg(W, H, "Hybrid quantization: two-stage re-scoring", desc, css, body) + + +# --------------------------------------------------------------------------- +# 2. Adaptive depth +# --------------------------------------------------------------------------- + +def adaptive_depth(): + rng = random.Random(10) + k, fixed, margin = 10, 30, 0.045 + # Codec scores for two queries: one with a crowded top, one well separated. + crowded = sorted((0.80 - abs(rng.gauss(0, 0.04)) for _ in range(70)), reverse=True) + separated = sorted((0.80 - abs(rng.gauss(0, 0.18)) for _ in range(70)), reverse=True) + + W, H = 820, 446 + x0, x1 = 50, 790 + lo, hi = 0.33, 0.81 + X = lambda s: x0 + (s - lo) / (hi - lo) * (x1 - x0) + dur = 12 + css, body = "", "" + body += 'Adaptive depth: re-score every candidate within λσ of the 10th score\n' + c, b = captions("ac", dur, [ + (0, 30, "The code's score noise σ is calibrated once per collection, on self-queries."), + (30, 62, "The margin λσ is the same for every query; how many candidates fall inside it is not."), + (62, 100, "A fixed shortlist (dashed) over-fetches for one query and under-fetches for the other."), + ], y=60) + css += c + body += b + + css += phase("grow", dur, [(0, "transform:scaleX(0)"), (30, "transform:scaleX(0)"), (52, "transform:scaleX(1)"), + (94, "transform:scaleX(1)"), (100, "transform:scaleX(0)")]) + css += phase("show", dur, [(0, "opacity:0"), (52, "opacity:0"), (56, "opacity:1"), (94, "opacity:1"), (100, "opacity:0")]) + css += phase("fix", dur, [(0, "opacity:0"), (62, "opacity:0"), (66, "opacity:1"), (94, "opacity:1"), (100, "opacity:0")]) + css += f".grow {{ transform-box: fill-box; transform-origin: right; animation: grow {dur}s infinite; }}\n" + css += f".show {{ opacity:1; animation: show {dur}s infinite; }}\n" + css += f".fix {{ opacity:1; animation: fix {dur}s infinite; }}\n" + + def panel(scores, top, name): + out = "" + kth = scores[k - 1] + inside = sum(1 for s in scores if s >= kth - margin) + base = top + 112 + out += f'{name}\n' + out += f'\n' + out += (f'\n') + # Dots in lanes; candidates inside the margin turn blue once the band has grown. + for i, s in enumerate(scores): + y = top + 24 + (i % 6) * 15 + if s >= kth - margin and i >= k: + n2 = f"in{name[:3]}{i}" + css_dot = phase(n2, dur, [(0, "fill:var(--other);fill-opacity:.55"), (52, "fill:var(--other);fill-opacity:.55"), + (56, "fill:var(--s1);fill-opacity:1"), (94, "fill:var(--s1);fill-opacity:1"), + (100, "fill:var(--other);fill-opacity:.55")]) + nonlocal_css.append(css_dot) + out += f'\n' + elif i < k: + out += f'\n' + else: + out += f'\n' + out += f'\n' + out += f'10th\n' + fx = X(scores[fixed - 1]) + out += f'\n' + out += f'fixed N = {fixed}\n' + out += f're-scores {inside} candidates\n' + return out, inside + + nonlocal_css = [] + p1, n1 = panel(crowded, 96, "Crowded query: many candidates score close to the 10th") + p2, n2 = panel(separated, 256, "Well-separated query: the 10th stands clear of the rest") + css += "".join(nonlocal_css) + body += p1 + p2 + lg = 412 + body += (f'the code\'s top 10' + f're-scored by margin' + f'not re-scored\n') + body += (f'Schematic: synthetic code scores, one σ for both queries. ' + 'Measured results: experiments/2026-11-hybrid-adaptive-matrix.\n') + desc = (f"Animated schematic of adaptive re-scoring depth. Two queries share one noise-based margin below their 10th " + f"code score. The crowded query has {n1} candidates inside the margin, the well-separated one {n2}; a fixed " + f"shortlist of {fixed} would under-fetch for the first and over-fetch for the second.") + return svg(W, H, "Adaptive depth: a per-query margin instead of a fixed shortlist", desc, css, body) + + +# --------------------------------------------------------------------------- +# 3. Adaptive bits (real data) +# --------------------------------------------------------------------------- + +def adaptive_bits(): + data = json.loads((HERE / "scifact-bge-small-alloc48.json").read_text()) + var, plain, wt = data["var"], data["plain"], data["weighted"] + d, group = len(var), 8 + groups = d // group + states = [ + ("uniform", [1] * d, "Uniform: 1 bit on every dimension (binary, 48 B)", "0.611"), + ("plain", plain, "mixed48-pca: rotate to principal components, spend bits where the variance is", "0.694"), + ("weighted", wt, "mixed48-pca-w: weight variance by how much energy queries put there", "0.703"), + ] + sums = {name: [sum(bits[g * group:(g + 1) * group]) for g in range(groups)] for name, bits, _, _ in states} + zero = {name: sum(1 for b in bits if b == 0) for name, bits, _, _ in states} + + W, H = 820, 486 + x0, x1 = 64, 790 + bw = (x1 - x0) / groups + top, base = 180, 370 + ymax = 40 + Y = lambda v: base - v / ymax * (base - top) + dur = 15 + css, body = "", "" + body += 'Adaptive bits: the same 48 bytes, spent where they matter\n' + + # Variance spectrum, its own small chart (one axis), log scale. + import math + vt, vb = 70, 138 + # Clip at 0.001: one near-dead component would otherwise stretch the axis to 1e-5. + lmin = -3 + lv = [max(math.log10(max(v, 1e-12)), lmin) for v in var] + VY = lambda l: vb - (l - lmin) / (0 - lmin) * (vb - vt) + VX = lambda i: x0 + (i + 0.5) / d * (x1 - x0) + pts = " ".join(f"{VX(i):.1f},{VY(lv[i]):.1f}" for i in range(d)) + for t in range(lmin, 1): + body += f'' + body += f'{10 ** t:g}\n' + body += f'\n' + body += f'variance per principal component, relative to the first (log scale, clipped at 0.001)\n' + + # Bars: total bits per group of 8 components, morphing between states. + for g in range(0, ymax + 1, 8): + body += f'' + body += f'{g}\n' + body += f'bits per group of 8 components (48 bytes = 384 bits in all)\n' + seq = ["uniform", "plain", "weighted"] + # 0–28% uniform, 33–61% plain, 66–94% weighted, back to uniform. + for gi in range(groups): + s = [sums[n][gi] / ymax for n in seq] + name = f"b{gi}" + stops = [(0, f"transform:scaleY({s[0]:.4f})"), (28, f"transform:scaleY({s[0]:.4f})"), + (33, f"transform:scaleY({s[1]:.4f})"), (61, f"transform:scaleY({s[1]:.4f})"), + (66, f"transform:scaleY({s[2]:.4f})"), (94, f"transform:scaleY({s[2]:.4f})"), + (100, f"transform:scaleY({s[0]:.4f})")] + css += phase(name, dur, stops) + css += f".{name} {{ transform-box: fill-box; transform-origin: bottom; transform: scaleY({s[2]:.4f}); animation: {name} {dur}s infinite; }}\n" + body += f'\n' + body += f'\n' + body += f'component 1' + body += f'component 384\n' + + items = [] + spans = [(0, 30), (31, 63), (64, 100)] + for (name, _, label, rec), (a, b) in zip(states, spans): + items.append((a, b, f"{label}")) + c1, b1 = captions("bc", dur, items, y=410, cls="cap") + stats = [] + for (name, _, _, rec), (a, b) in zip(states, spans): + stats.append((a, b, f"recall@10 without re-scoring: {rec} · components with 0 bits: {zero[name]} of {d}")) + c2, b2 = captions("bs", dur, stats, y=432, cls="num") + css += c1 + c2 + body += b1 + b2 + # The query-weighted state puts a few bits back on the last components: low variance, but + # they carry the corpus's shared mean direction, which every query projects onto. + css += phase("tail", dur, [(0, "opacity:0"), (66, "opacity:0"), (70, "opacity:1"), (94, "opacity:1"), (100, "opacity:0")]) + css += f".tail {{ opacity:1; animation: tail {dur}s infinite; }}\n" + body += (f'bits return to the last 44 components: little variance,' + f'but 28% of the corpus mean, which every query points along\n') + body += ('Real data: SciFact, bge-small-en-v1.5, 8,036 chunks, 300 claims. pq48 at the same 48 bytes reaches 0.711.' + 'Mixed precision by importance is the idea behind structure-aware MoE quantization (QuantMoE-Bench).\n') + desc = ("Animated bar chart of how a 48-byte budget is spread over 384 principal components of SciFact's bge-small " + f"embeddings. Uniform binary gives every component 1 bit (recall@10 0.611). Variance-based allocation " + f"concentrates bits on the leading components and gives {zero['plain']} components none (0.694); " + f"query-weighted allocation goes further, {zero['weighted']} components with none (0.703).") + return svg(W, H, "Adaptive bits: mixed-precision allocation at 48 bytes", desc, css, body) + + +if __name__ == "__main__": + for name, fn in [("hybrid-rescoring.svg", hybrid), ("adaptive-depth.svg", adaptive_depth), ("adaptive-bits.svg", adaptive_bits)]: + (HERE / name).write_text(fn()) + print("wrote", HERE / name) diff --git a/docs/assets/animations/scifact-bge-small-alloc48.json b/docs/assets/animations/scifact-bge-small-alloc48.json new file mode 100644 index 0000000..1e89078 --- /dev/null +++ b/docs/assets/animations/scifact-bge-small-alloc48.json @@ -0,0 +1 @@ +{"var": [1.0, 0.39134126439623, 0.3496060513273873, 0.31595562269325783, 0.27839887791768925, 0.2560764744407206, 0.2106557348265734, 0.20141261230956678, 0.17205700091762918, 0.1694689781684781, 0.15670643906039008, 0.14818463943416052, 0.13528590609216548, 0.133632168854869, 0.12713526134315942, 0.12038752565574859, 0.11740274955546613, 0.11612737530841535, 0.10966765030880443, 0.10022672950990126, 0.09597221552830504, 0.09416645470354275, 0.0897062733654618, 0.08829125668874234, 0.0865481361631404, 0.08514376604622365, 0.08207708583300333, 0.07979409035062336, 0.07869511441982824, 0.07687102383040244, 0.07481058615818921, 0.07264456168849127, 0.07091941954939518, 0.06898615151401433, 0.0673025322873146, 0.06611170096068165, 0.06496654048023187, 0.06463273495895579, 0.062363045077972225, 0.06168799384286252, 0.06078482019050579, 0.05909247284983816, 0.05819816407514659, 0.057162722344408154, 0.05666445379314992, 0.055642514308145825, 0.054891234148445274, 0.05464257643109169, 0.05298220648579837, 0.052255319312683596, 0.050229837084433133, 0.049928403768241376, 0.049495922827931776, 0.048706883410482946, 0.048070291750025265, 0.04743586360155167, 0.04703310095479999, 0.04606850090748441, 0.04562184553180973, 0.04481238780065704, 0.04470614791945737, 0.043831648633395226, 0.04348099873273696, 0.043185493702070726, 0.042564347857302544, 0.0418145926307804, 0.041541618240013116, 0.040959217618167745, 0.04048983107135532, 0.04013877227408187, 0.03932441481258852, 0.03890798896848211, 0.03851867123239437, 0.038119319522703435, 0.037648007029831586, 0.03736179884260297, 0.03705055938897124, 0.03694661464816095, 0.036193931557568455, 0.035823388949576956, 0.03559611055864424, 0.03520842299696664, 0.035023113788243534, 0.034573018064219564, 0.03442555515236499, 0.033838931935893046, 0.033737953301576744, 0.0332053037675544, 0.03290633081644007, 0.032697393024546204, 0.032282294790795994, 0.03215241790104038, 0.03161307317735745, 0.031261931277445955, 0.030929248366534455, 0.030791728025711698, 0.030627936493268827, 0.03058120733637114, 0.030174571329789134, 0.02980193183276241, 0.02954288861957916, 0.029369533540991623, 0.028954133478995486, 0.02875126260361623, 0.02864234904438538, 0.028382232251897867, 0.028265182491288225, 0.027611630299355074, 0.027283102610038423, 0.026974494350990513, 0.026680319408314404, 0.026595889019600694, 0.026410906059024155, 0.026254603004913026, 0.026080166592958882, 0.025944636072886388, 0.025807118940548615, 0.025646206976489925, 0.025507308059213595, 0.02532482132649611, 0.025091225055296998, 0.02481734159697431, 0.0247182052511134, 0.024456295140222305, 0.024107062685350074, 0.023940666679640683, 0.02365848807218462, 0.023343360656366097, 0.02326051767025958, 0.02304732943333772, 0.022971029239951417, 0.02285096169449531, 0.02257667189459206, 0.02250770686220711, 0.022362210892392393, 0.02222430593143528, 0.022146058964851483, 0.022117423589758715, 0.022003785634970907, 0.021871522646568428, 0.021784814932734573, 0.021581083371064385, 0.021154111565228586, 0.021039979416007284, 0.020979723568142847, 0.020821471968335163, 0.020686814264570377, 0.020504686883530313, 0.020386891998983317, 0.020205310311146998, 0.020156252813216386, 0.020061090435095564, 0.019832417763675707, 0.019717340439953976, 0.01960595220039487, 0.019550225612816476, 0.01930417186580708, 0.01920903796601698, 0.019118348146219943, 0.018936513611833933, 0.01886449228880342, 0.01877710052075384, 0.018731855992431726, 0.018573200303693145, 0.01840371617893902, 0.01834322498490772, 0.01823057870226212, 0.018143878629314183, 0.018099479430126873, 0.017803139864334107, 0.017707690876069108, 0.017667780705379617, 0.017536074683726168, 0.0173161695860729, 0.017267670257651205, 0.01713406883872043, 0.01702702305372836, 0.016935927306379578, 0.016789447939799054, 0.016618970014251565, 0.0165605344481448, 0.016468454126418404, 0.016356061807699284, 0.016227912854336765, 0.016164161403722337, 0.016023441552851524, 0.015974556881522647, 0.01592741839249819, 0.01579390099607421, 0.01563823754203696, 0.015500089523595089, 0.01536352840169009, 0.015281913722668329, 0.015221240680803938, 0.015069686999128577, 0.01501600213250068, 0.014959732589477687, 0.01473677198653329, 0.01471857906253492, 0.014548472089878481, 0.014359450213777157, 0.014326718627607716, 0.01426966063161707, 0.01414185526332443, 0.014115840275000844, 0.014054441634707164, 0.014024143721019811, 0.01386626074311244, 0.013800739332965157, 0.013729491120792161, 0.01366693004197577, 0.013539004015227684, 0.013490713114855852, 0.013432350265979578, 0.013347617019144328, 0.01325644326794014, 0.013201895052233226, 0.013091836558617804, 0.013052781201159958, 0.012986204944295308, 0.01286341658777245, 0.012753444705275957, 0.012668021423633264, 0.012532796468601403, 0.012460059702106613, 0.012435781313837645, 0.01235964394030797, 0.012345126158184906, 0.012204695289386584, 0.012132188359035484, 0.012121417959160868, 0.012030577365429234, 0.011952795890848073, 0.01186615106573008, 0.011745869500282637, 0.01169428189208853, 0.011633996441412914, 0.011593076381497676, 0.011461766336611775, 0.011428690893860947, 0.011375626641593138, 0.011346628087151573, 0.011307391385062291, 0.011277279597740353, 0.011183149861353292, 0.01114832253650065, 0.01105608459782311, 0.01099515916881342, 0.010914850257935597, 0.010842722441454766, 0.010750481223547735, 0.010676872780639479, 0.010587711726527296, 0.010539337137236737, 0.010515128943463924, 0.010450450920191146, 0.010407051688116436, 0.01036583328507681, 0.010277255453496247, 0.010256700747482307, 0.010202886922511646, 0.010092985351844741, 0.010009442526184575, 0.00998169372609048, 0.009947514930201662, 0.009826871073062194, 0.00973106766376329, 0.00969645324182059, 0.009680648469279499, 0.009643878327822184, 0.009596587637293796, 0.009509202039516886, 0.009466200377119746, 0.00937385323293766, 0.009358364533570916, 0.009262808536093988, 0.009193431818542765, 0.009142849991138517, 0.009097424943512667, 0.009085658466338208, 0.009022218449408518, 0.008958274735340659, 0.008887716813467466, 0.008831053665017989, 0.008774044616926487, 0.008705662613762273, 0.008644436499713614, 0.008639921776876253, 0.008569510888579664, 0.008520013163906777, 0.008454577441738224, 0.008419553380696693, 0.00839470000954221, 0.00824977097009384, 0.008196592564584226, 0.008170327488352799, 0.008099281365090414, 0.00803613047816981, 0.008012593855526927, 0.007937095512906025, 0.007906309421039402, 0.007856957161811553, 0.0077793146186748525, 0.007728136554942177, 0.007672670210567666, 0.007635557900921481, 0.007608166220098032, 0.007566589305614287, 0.007490117092045346, 0.007439563579708671, 0.007414872823850272, 0.007329654261822421, 0.007299932999050368, 0.007212300015075819, 0.007191620087145013, 0.007143819623201941, 0.007056348284034112, 0.007021928117747978, 0.007005581232133432, 0.006948925204175803, 0.006866356246196426, 0.006827068422472897, 0.006773397779998446, 0.006725508714276735, 0.0066913394341998516, 0.006652526252912546, 0.006534035830982947, 0.006504680905935383, 0.006416122069632486, 0.006352072725854465, 0.0063227506840388276, 0.006261661931085398, 0.006210332442922852, 0.006188694622778561, 0.0061318958859779915, 0.006034261318434921, 0.0059822596673720614, 0.00595127886842367, 0.005925083249668537, 0.0058295979333348215, 0.005816884566962228, 0.005786337865382163, 0.005744825114543283, 0.0056742491985402495, 0.005600899231190342, 0.005566691836066337, 0.005514749526665931, 0.005502555590552398, 0.0054526641024227876, 0.005345785494200853, 0.00532774530326598, 0.005309388567924551, 0.005292482920037143, 0.005211607133828315, 0.005176766484914195, 0.00506441478672639, 0.005002617811967771, 0.004924455416518881, 0.004875140427058741, 0.004816960179373682, 0.0047096536708319285, 0.0046893844868070335, 0.0046225668244471285, 0.00451007000563216, 0.0043986096021096655, 0.004349770898096713, 0.004287748275926356, 0.004219578560474761, 0.0041545132755926775, 0.0040336298316401035, 0.0039501433610620185, 0.003926478815471053, 0.00382689836365158, 0.0037231473996302677, 0.003509502213595666, 0.003052606608379145, 0.0029685403497338976, 0.0026763607607460556, 0.002135596994617984, 0.0016904888134794548, 0.0006258427297054932, 0.0005264920570519339, 5.4716114804062615e-05, 1.73375150199403e-09], "plain": [4, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], "weighted": [6, 5, 5, 5, 4, 4, 4, 4, 4, 4, 3, 3, 3, 3, 3, 4, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 2, 3, 3, 3, 3, 2, 2, 2, 2, 2, 2, 2, 2, 2, 3, 2, 3, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 3, 2, 2, 2, 3, 2, 2, 2, 1, 2, 1, 1, 2, 2, 2, 2, 2, 2, 1, 1, 1, 1, 1, 2, 2, 2, 1, 2, 1, 1, 1, 1, 1, 1, 2, 2, 1, 1, 1, 2, 1, 2, 2, 2, 1, 2, 2, 1, 1, 1, 1, 1, 1, 2, 1, 1, 1, 1, 1, 1, 1, 2, 1, 2, 1, 1, 1, 1, 1, 1, 2, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 1, 2, 1, 0, 1, 1, 0, 1, 0, 1, 2, 0, 1, 1, 2, 1, 1, 1, 2, 0, 0, 1, 0, 0, 0, 0, 1, 0, 2, 1, 1, 1, 0, 2, 1, 1, 0, 0, 1, 2, 1, 0, 2, 1, 0, 1, 2, 1, 0, 0, 1, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 1, 0, 1, 1, 0, 2, 0, 0, 2, 0, 0, 0, 0, 0, 1, 1, 1, 0, 1, 0, 0, 1, 0, 1, 0, 0, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 2, 0, 0, 1, 0, 0, 1, 0, 0, 2, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 2, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 1, 1, 0, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 0, 0, 0, 0, 0, 1, 1, 2, 2, 0, 0, 0, 0, 0, 0]} \ No newline at end of file diff --git a/experiments/2026-11-hybrid-adaptive-matrix/README.md b/experiments/2026-11-hybrid-adaptive-matrix/README.md new file mode 100644 index 0000000..235c5c7 --- /dev/null +++ b/experiments/2026-11-hybrid-adaptive-matrix/README.md @@ -0,0 +1,170 @@ +# The hybrid × adaptive matrix + +Two ways to make compression adaptive, crossed with the hybrid (two-stage) retrieval from the +[re-scoring experiment](../2026-11-two-stage-rescoring/): + +- **Adaptive bits.** Spend the scan code's bytes unevenly. `mixed` gives each dimension 0 to 8 + bits by greedy reverse water-filling on its variance. `-pca` first rotates into the corpus's + principal components, so the variance has somewhere to concentrate. `-w` weights each + component by how much energy queries put into it, so the allocation minimises score error + rather than reconstruction error. This is the embedding analogue of structure-aware, + mixed-precision quantization of mixture-of-experts models ([QuantMoE-Bench][quantmoe]), where + the blocks and experts that matter more get more bits. +- **Adaptive depth.** Re-score every candidate whose scan score is within λσ of the scan's own + 10th score, where σ is the scan code's noise, calibrated once per collection on self-queries. + Crowded queries get deep re-scoring, well-separated ones shallow. + +**Hypothesis.** +1. With a PCA rotation, adaptive bits beat uniform sign bits at equal bytes and approach + product quantization. Without the rotation they do little, because raw embedding dimensions + have fairly even variance. +2. At the same mean number of re-scored candidates, adaptive depth beats a fixed depth, + mostly by rescuing the worst queries. + +**Result.** The first is confirmed, more strongly than expected. The second is confirmed only +in its weak form: the gain is real and almost always positive, but small. +- **Adaptive bits beat PQ on average.** `mixed48-pca-w` keeps 0.768 of the exact top 10 with no + re-scoring, against 0.747 for `pq48` and 0.669 for `binary`. It wins at 24 and 96 bytes too. + The gain is all on MiniLM (+4.4 points); on bge-small the two tie (0.723 vs 0.724), and `pq48` + keeps a slightly smaller nDCG loss (0.0160 vs 0.0177). +- **The rotation is what makes it work.** Without it, `mixed48` (0.696) is no better than + `binary-centred` (0.698). +- **Adaptive depth beats fixed depth at equal mean depth, but only by 0.25 to 0.6 points of + recall@10.** It is ahead in 156 of 180 comparisons. Its gain is larger for the worst queries: + worst-decile recall rises by 4.1 points at a 1σ margin. The price is variable work: the + 90th-percentile query re-scores 1.3 to 2.1× the mean. *Correction:* the first version of this + write-up reported +1.5 points of recall (and +3.7 worst-decile) from comparing against fixed + depths of only 20, 50 and 100. Interpolating between such sparse points runs below the true + fixed-depth curve, which bends, and so flattered the margin policy by about 3×. The numbers + here use the per-query traces from the [routing experiment](../2026-11-routing-oracle/), + which score a fixed depth at exactly the margin's mean depth. +- **The two combine.** `mixed48-pca-w` with a 1σ margin re-scores 19 candidates on average and + keeps 0.929 of the top 10. `binary` with a fixed 20 keeps 0.841 for the same bytes. + +## Setup + +| | | +|---|---| +| Collections | The ten from [neighbour spacing](../2026-11-neighbour-spacing/): SciFact, NFCorpus, ArguAna, CoSQA, FiQA × bge-small-en-v1.5, all-MiniLM-L6-v2 (384 dims, 200-word chunks, 6,260 to 71,274 per collection) | +| Queries | Each set's judged test queries (300 to 1,406) | +| Scan codecs | Uniform: `binary`, `binary-centred` (48 B), `pq24`, `pq48`, `pq96`. Adaptive: `mixed48` (raw dimensions), `mixed24-pca`, `mixed48-pca`, `mixed24-pca-w`, `mixed48-pca-w`, `mixed96-pca-w` | +| Second stage | `int8` (392 B) or `f32` (1,536 B), read for candidates only | +| Policies | none; fixed depth 20, 50, 100; margin 1σ, 2σ, 3σ (capped at 1,000) | +| Calibration | σ = the scan code's rank noise on 200 self-queries per collection. The test queries' exact scores never set their own depth | +| Fitting | Trained codecs fitted on every vector of the collection. PQ: 256 centroids per subspace, 20 Lloyd iterations. `mixed`: PCA by Jacobi on up to 20,000 vectors | +| Metrics | recall@10 against exact f32; 10th-percentile query recall@10 (worst decile); nDCG@10 loss against the f32 ceiling, from the relevance judgments; mean and p90 depth | +| Search | brute force, no index | +| Hardware | Apple M4 Pro, release build | +| Code | hvec commit `6113ab7` (`bench rescore` with policies, `mixed`) | +| Command | `./run.sh` (reuses the spacing experiment's collections; builds them if missing) | + +## Results + +All numbers are means over the ten collections. The full matrix, 143 cells per collection, is in +`results/summary.txt`; the raw reports are `results/--k10.json`. + +### The matrix: recall@10 by scan codec and re-scoring policy (f32 second stage) + +| scan codec | bytes | none | fixed 20 | fixed 50 | fixed 100 | margin 1σ (mean depth) | margin 2σ (mean depth) | nDCG loss, none → fixed 50 | +|---|---:|---:|---:|---:|---:|---:|---:|---| +| `binary` | 48 | 0.669 | 0.841 | 0.948 | 0.981 | 0.869 (23) | 0.963 (52) | 0.0240 → 0.0003 | +| `binary-centred` | 48 | 0.698 | 0.869 | 0.966 | 0.991 | 0.911 (26) | 0.984 (65) | 0.0236 → 0.0002 | +| `mixed48` | 48 | 0.696 | 0.868 | 0.966 | 0.991 | 0.909 (26) | 0.983 (65) | 0.0242 → 0.0000 | +| `pq48` | 48 | 0.747 | 0.912 | 0.986 | 0.998 | 0.924 (21) | 0.985 (41) | 0.0160 → 0.0002 | +| `mixed48-pca` | 48 | 0.755 | 0.920 | 0.989 | 0.998 | 0.919 (19) | 0.982 (35) | 0.0167 → 0.0000 | +| `mixed48-pca-w` | 48 | **0.768** | **0.931** | **0.991** | **0.999** | **0.929 (19)** | 0.983 (36) | 0.0177 → 0.0000 | +| `pq24` | 24 | 0.606 | 0.779 | 0.918 | 0.968 | 0.887 (42) | 0.979 (144) | 0.0586 → 0.0046 | +| `mixed24-pca` | 24 | 0.631 | 0.807 | 0.937 | 0.977 | 0.880 (31) | 0.971 (89) | 0.0497 → 0.0037 | +| `mixed24-pca-w` | 24 | **0.635** | **0.815** | **0.937** | **0.977** | 0.888 (33) | 0.973 (100) | 0.0523 → 0.0047 | +| `pq96` | 96 | 0.870 | 0.986 | 1.000 | 1.000 | 0.958 (14) | 0.990 (18) | 0.0031 → −0.0001 | +| `mixed96-pca-w` | 96 | **0.886** | **0.990** | 1.000 | 1.000 | **0.963 (14)** | **0.991 (18)** | 0.0022 → 0.0000 | + +An int8 second stage matches f32 to within a point up to a depth of about 50, then levels off +at int8's own recall. At depth 100: `binary` 0.974 against 0.981, `pq48` 0.989 against 0.998, `mixed48-pca-w` +0.990 against 0.999. + +### Adaptive bits alone, by byte budget (no re-scoring) + +| bytes | codec | recall@10 | bge-small | MiniLM | worst decile | nDCG loss | +|---:|---|---:|---:|---:|---:|---:| +| 24 | `mixed24-pca-w` | **0.635** | 0.592 | **0.677** | 0.37 | 0.052 | +| 24 | `mixed24-pca` | 0.631 | **0.596** | 0.667 | 0.37 | **0.050** | +| 24 | `pq24` | 0.606 | 0.587 | 0.624 | 0.33 | 0.059 | +| 48 | `mixed48-pca-w` | **0.768** | 0.723 | **0.813** | **0.55** | 0.018 | +| 48 | `mixed48-pca` | 0.755 | 0.717 | 0.793 | 0.54 | 0.017 | +| 48 | `pq48` | 0.747 | **0.724** | 0.769 | 0.52 | **0.016** | +| 48 | `binary-centred` | 0.698 | 0.667 | 0.728 | 0.43 | 0.024 | +| 48 | `mixed48` (no rotation) | 0.696 | 0.663 | 0.729 | 0.43 | 0.024 | +| 48 | `binary` | 0.669 | 0.622 | 0.715 | 0.41 | 0.024 | +| 96 | `mixed96-pca-w` | **0.886** | 0.855 | **0.916** | **0.74** | **0.002** | +| 96 | `pq96` | 0.870 | **0.855** | 0.885 | 0.73 | 0.003 | + +### Adaptive depth against fixed depth at the same mean depth + +With an f32 second stage, a query's recall at any depth is the share of its true neighbours the +scan ranks within that depth. The routing experiment's traces record those ranks for six of the +scan codecs here (`binary`, `binary-centred`, `pq24`, `pq48`, `mixed24-pca-w`, `mixed48-pca-w`) +on all ten collections. So a fixed depth can be scored at exactly the margin policy's mean +depth, interpolating only between neighbouring whole numbers. + +| margin | comparisons | mean depth | Δ recall@10 | ahead in | Δ worst decile | p90 depth / mean depth | +|---|---:|---:|---:|---:|---:|---:| +| 1σ | 60 | 27 | +0.006 | 48 / 60 | **+0.041** | 1.29–1.60 | +| 2σ | 60 | 73 | +0.006 | 59 / 60 | +0.020 | 1.39–1.94 | +| 3σ | 60 | 163 | +0.003 | 49 / 60 | +0.007 | 1.49–2.09 | + +The nDCG change is zero to three decimals at every margin, in the matrix's own cells. Section 3 of `results/summary.txt` is the original, sparse comparison, kept for the +record; it overstates the gain for the reason above. + +### Benefits and drawbacks of each combination + +Against the plain 48-byte `binary` scan with no re-scoring (recall@10 0.669, worst decile 0.41, +nDCG loss 0.024). Memory is per vector: what is scanned in RAM, plus what is fetched for +candidates. + +| combination | memory per vector | recall@10 | worst decile | nDCG loss | benefits | drawbacks | +|---|---|---:|---:|---:|---|---| +| `binary`, no re-score | 48 B | 0.669 | 0.41 | 0.024 | Stateless, nothing to fit | Loses a third of the top 10; its asymmetric score is no faster than f32 | +| `pq48`, no re-score | 48 B (+ codebooks) | 0.747 | 0.52 | 0.016 | Best nDCG at 48 B with no re-score | k-means fit per collection; codebooks to store; refit when the corpus drifts | +| `mixed48-pca-w`, no re-score | 48 B (+ a 384 × 384 rotation) | 0.768 | 0.55 | 0.018 | Best recall at 24, 48 and 96 B; PCA fit, no iterative clustering | Ties PQ on bge-small and loses slightly on nDCG; its scan is 2.2× slower per vector than f32 ([zero-copy scan](../2026-11-zero-copy-scan/)); refit on drift | +| hybrid, fixed depth, int8 | 48 B RAM + 392 B disk | 0.944–0.983 at depth 50 | 0.84–0.97 | ≈ 0 | Second store 4× smaller than f32 | Levels off at int8's own 0.99; random reads per query | +| hybrid, fixed depth, f32 | 48 B RAM + 1,536 B disk | 0.948–0.991 at depth 50 | 0.86–0.98 | ≈ 0 | Reaches exact; the depth is predictable from noise ÷ band | Keeps a full f32 copy; N random reads per query | +| hybrid, adaptive depth | as above | +0.003 to +0.006 over fixed at equal mean depth | +0.007 to +0.041 | ≈ 0 | Fewer bad queries for the same average work | A small gain for variable work per query (p90 1.3–2.1× the mean); σ calibrated per collection; one more parameter | +| adaptive bits + adaptive depth (`mixed48-pca-w`, 1σ) | 48 B RAM + 1,536 B disk | 0.929 at mean depth 19 | | ≈ 0 | The best recall per candidate re-scored of any 48-byte scan | Every drawback above together: two fits, a rotation and variable depth | + +## Discussion + +**PCA plus bit allocation is a serious alternative to PQ.** It never trails `pq` by more than +a point of recall at equal bytes, and it leads by 3 to 5 points on MiniLM. It fits with one +eigendecomposition instead of k-means, and the query-weighted allocation (`-w`) is worth about +another point at 48 bytes. Its nDCG at 48 bytes is slightly worse than PQ's despite the better +recall, which says the neighbours it recovers are not always the judged ones. That is the same +pattern the labelled SciFact experiment found for centring. + +**Why MiniLM gains and bge-small does not.** Both models have a PCA spectrum to exploit, but on +bge-small (measured on SciFact) 60% of an average vector's energy is the shared mean direction. +PQ's codebooks probably absorb that common component for free, while a quantizer working per +principal component has to spend bits on it. The `-w` allocation does spend bits on it, which is the bump of bits on the +last components in the [README animation](../../docs/assets/animations/adaptive-bits.svg). +Centring the rotation, or giving the mean its own exact term, is the obvious next variant. + +**Adaptive depth is a small, reliable win, not a large one.** Half a point of recall at equal +work, up to four points for the worst decile of queries, and rarely negative. The +[routing experiment](../2026-11-routing-oracle/) asks how much more a better router could get. +A hindsight oracle shows about 6 points of headroom, but the band a router can see from the +scan's own scores tracks the true band only moderately (Spearman 0.52 to 0.70), and no router +built on it captured more than 6% of the headroom. + +**Caveats.** +- **Recall is against exact f32, not the answer.** nDCG uses binary relevance. Whether any of + this reaches a chat model's answer is Experiment 1. +- **No fit or encode times.** The cost of fitting the PCA and the PQ codebooks is not measured + here; the [zero-copy scan](../2026-11-zero-copy-scan/) experiment times the scans. +- **Parameters not counted in bytes per vector.** The rotation is 590 KB per collection, and + PQ's codebooks are of similar size, as before. +- **Margin cells with few comparisons.** 3σ cells often exceed the measured fixed range, so + they're compared on few collections. +- **The same collections as the spacing experiment**, including CoSQA's duplicates and + bge-small without its query instruction. + +[quantmoe]: https://github.com/UNITES-Lab/moe-quantization diff --git a/experiments/2026-11-hybrid-adaptive-matrix/results/arguana-bge-small-k10.json b/experiments/2026-11-hybrid-adaptive-matrix/results/arguana-bge-small-k10.json new file mode 100644 index 0000000..08d09da --- /dev/null +++ b/experiments/2026-11-hybrid-adaptive-matrix/results/arguana-bge-small-k10.json @@ -0,0 +1,3900 @@ +{ + "cells": [ + { + "bytes_read_per_query": 539136.0, + "depth": 0, + "doc_recall_at_k": 0.7802275960170697, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.3929241125565956, + "p10_recall_at_k": 0.5, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 1406, + "recall_at_k": 0.7481507823613063, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.9943100995732574, + "vectors": 11232 + }, + { + "bytes_read_per_query": 546976.0, + "depth": 20, + "doc_recall_at_k": 0.817923186344239, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.42094676051288715, + "p10_recall_at_k": 0.8, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9161450924608824, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 558736.0, + "depth": 50, + "doc_recall_at_k": 0.8172119487908962, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.4208425207456014, + "p10_recall_at_k": 0.9, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9766002844950221, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 578336.0, + "depth": 100, + "doc_recall_at_k": 0.8186344238975818, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.4212198358866503, + "p10_recall_at_k": 0.9, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9881223328591763, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 546314.3954480797, + "depth": 18, + "doc_recall_at_k": 0.8193456614509246, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 18.312233285917497, + "metric": "cosine", + "ndcg_at_k": 0.4214881624853719, + "p10_recall_at_k": 0.8, + "p90_depth": 26, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 1406, + "recall_at_k": 0.9094594594594584, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 552901.4452347084, + "depth": 35, + "doc_recall_at_k": 0.8172119487908962, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 35.115931721194876, + "metric": "cosine", + "ndcg_at_k": 0.42073541088822325, + "p10_recall_at_k": 0.9, + "p90_depth": 61, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 1406, + "recall_at_k": 0.9742532005689917, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 565846.3669985775, + "depth": 68, + "doc_recall_at_k": 0.8165007112375533, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 68.13869132290185, + "metric": "cosine", + "ndcg_at_k": 0.4205195035759079, + "p10_recall_at_k": 0.9, + "p90_depth": 133, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 1406, + "recall_at_k": 0.9885490753911819, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 569856.0, + "depth": 20, + "doc_recall_at_k": 0.8193456614509246, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.42153573564741575, + "p10_recall_at_k": 0.8, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9196301564722622, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 615936.0, + "depth": 50, + "doc_recall_at_k": 0.817923186344239, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.42116155692232027, + "p10_recall_at_k": 0.9, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9834281650071123, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 692736.0, + "depth": 100, + "doc_recall_at_k": 0.8193456614509246, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.42158630990448837, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9963726884779516, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 567263.5903271693, + "depth": 18, + "doc_recall_at_k": 0.8200568990042674, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 18.312233285917497, + "metric": "cosine", + "ndcg_at_k": 0.42181437980732245, + "p10_recall_at_k": 0.8, + "p90_depth": 26, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 1406, + "recall_at_k": 0.9118065433854898, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 593074.0711237553, + "depth": 35, + "doc_recall_at_k": 0.8186344238975818, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 35.115931721194876, + "metric": "cosine", + "ndcg_at_k": 0.4212983910673265, + "p10_recall_at_k": 0.9, + "p90_depth": 61, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 1406, + "recall_at_k": 0.9809388335704136, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 643797.0298719773, + "depth": 68, + "doc_recall_at_k": 0.8172119487908962, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 68.13869132290185, + "metric": "cosine", + "ndcg_at_k": 0.420885977593746, + "p10_recall_at_k": 1.0, + "p90_depth": 133, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 1406, + "recall_at_k": 0.9968705547652915, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 539136.0, + "depth": 0, + "doc_recall_at_k": 0.7930298719772404, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.40185348712965036, + "p10_recall_at_k": 0.6, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 1406, + "recall_at_k": 0.7726173541962991, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.9871977240398293, + "vectors": 11232 + }, + { + "bytes_read_per_query": 546976.0, + "depth": 20, + "doc_recall_at_k": 0.8172119487908962, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.4210602304318324, + "p10_recall_at_k": 0.8, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9354907539118054, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 558736.0, + "depth": 50, + "doc_recall_at_k": 0.8186344238975818, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.4212339848284331, + "p10_recall_at_k": 0.9, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9852062588904712, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 578336.0, + "depth": 100, + "doc_recall_at_k": 0.8186344238975818, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.42116388796776005, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9899004267425333, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 546546.9189189189, + "depth": 19, + "doc_recall_at_k": 0.8157894736842105, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 18.905405405405407, + "metric": "cosine", + "ndcg_at_k": 0.4203989044091234, + "p10_recall_at_k": 0.8, + "p90_depth": 27, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 1406, + "recall_at_k": 0.9373399715504985, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 553693.5305832148, + "depth": 37, + "doc_recall_at_k": 0.8186344238975818, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 37.13655761024182, + "metric": "cosine", + "ndcg_at_k": 0.42120956925711833, + "p10_recall_at_k": 0.9, + "p90_depth": 65, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 1406, + "recall_at_k": 0.9853485064011401, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 568331.6358463727, + "depth": 74, + "doc_recall_at_k": 0.8186344238975818, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 74.47866287339971, + "metric": "cosine", + "ndcg_at_k": 0.4211553778899891, + "p10_recall_at_k": 1.0, + "p90_depth": 148, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 1406, + "recall_at_k": 0.9905405405405417, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 569856.0, + "depth": 20, + "doc_recall_at_k": 0.8172119487908962, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.42125593800881284, + "p10_recall_at_k": 0.8, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9396870554765278, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 615936.0, + "depth": 50, + "doc_recall_at_k": 0.8193456614509246, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.42160896892404215, + "p10_recall_at_k": 1.0, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9931721194879096, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 692736.0, + "depth": 100, + "doc_recall_at_k": 0.8193456614509246, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.4215397347670436, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9985064011379802, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 568174.7027027027, + "depth": 19, + "doc_recall_at_k": 0.8157894736842105, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 18.905405405405407, + "metric": "cosine", + "ndcg_at_k": 0.4205747615812793, + "p10_recall_at_k": 0.8, + "p90_depth": 27, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 1406, + "recall_at_k": 0.9413229018492176, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 596177.7524893314, + "depth": 37, + "doc_recall_at_k": 0.8200568990042674, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 37.13655761024182, + "metric": "cosine", + "ndcg_at_k": 0.42178163703478416, + "p10_recall_at_k": 1.0, + "p90_depth": 65, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 1406, + "recall_at_k": 0.9928165007112386, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 653535.226173542, + "depth": 74, + "doc_recall_at_k": 0.8193456614509246, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 74.47866287339971, + "metric": "cosine", + "ndcg_at_k": 0.42153122468927257, + "p10_recall_at_k": 1.0, + "p90_depth": 148, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 1406, + "recall_at_k": 0.9992887624466572, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 539136.0, + "depth": 0, + "doc_recall_at_k": 0.8015647226173542, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.40950995673475227, + "p10_recall_at_k": 0.7, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 1406, + "recall_at_k": 0.8109530583214749, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.9914651493598862, + "vectors": 11232 + }, + { + "bytes_read_per_query": 546976.0, + "depth": 20, + "doc_recall_at_k": 0.8165007112375533, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.4206369794377869, + "p10_recall_at_k": 0.9, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9633001422475114, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 558736.0, + "depth": 50, + "doc_recall_at_k": 0.817923186344239, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.42094978413016126, + "p10_recall_at_k": 0.9, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.989331436699859, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 578336.0, + "depth": 100, + "doc_recall_at_k": 0.817923186344239, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.42094978413016126, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9910384068278817, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 545394.0597439545, + "depth": 16, + "doc_recall_at_k": 0.817923186344239, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 15.96443812233286, + "metric": "cosine", + "ndcg_at_k": 0.42119477614048123, + "p10_recall_at_k": 0.8, + "p90_depth": 22, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 1406, + "recall_at_k": 0.9401849217638705, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 549314.6173541963, + "depth": 26, + "doc_recall_at_k": 0.8172119487908962, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 25.965860597439544, + "metric": "cosine", + "ndcg_at_k": 0.4207441903703335, + "p10_recall_at_k": 0.9, + "p90_depth": 43, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 1406, + "recall_at_k": 0.9830725462304432, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 555871.2773826458, + "depth": 43, + "doc_recall_at_k": 0.817923186344239, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 42.69203413940256, + "metric": "cosine", + "ndcg_at_k": 0.42094978413016126, + "p10_recall_at_k": 1.0, + "p90_depth": 80, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 1406, + "recall_at_k": 0.9903982930298733, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 569856.0, + "depth": 20, + "doc_recall_at_k": 0.8172119487908962, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.4210836397781796, + "p10_recall_at_k": 0.9, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9678520625889054, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 615936.0, + "depth": 50, + "doc_recall_at_k": 0.8186344238975818, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.4213256309294448, + "p10_recall_at_k": 1.0, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9978662873399715, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 692736.0, + "depth": 100, + "doc_recall_at_k": 0.8186344238975818, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.4213256309294448, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9998577524893316, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 563657.3769559033, + "depth": 16, + "doc_recall_at_k": 0.8172119487908962, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 15.96443812233286, + "metric": "cosine", + "ndcg_at_k": 0.42117089923073325, + "p10_recall_at_k": 0.8, + "p90_depth": 22, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 1406, + "recall_at_k": 0.9435988620199159, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 579019.5618776671, + "depth": 26, + "doc_recall_at_k": 0.817923186344239, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 25.965860597439544, + "metric": "cosine", + "ndcg_at_k": 0.4211333234077255, + "p10_recall_at_k": 1.0, + "p90_depth": 43, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 1406, + "recall_at_k": 0.9906827880512105, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 604710.9644381223, + "depth": 43, + "doc_recall_at_k": 0.8186344238975818, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 42.69203413940256, + "metric": "cosine", + "ndcg_at_k": 0.4213256309294448, + "p10_recall_at_k": 1.0, + "p90_depth": 80, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 1406, + "recall_at_k": 0.999217638691323, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 539136.0, + "depth": 0, + "doc_recall_at_k": 0.7901849217638691, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.40054506887771435, + "p10_recall_at_k": 0.6, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 1406, + "recall_at_k": 0.7702702702702678, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.9864864864864865, + "vectors": 11232 + }, + { + "bytes_read_per_query": 546976.0, + "depth": 20, + "doc_recall_at_k": 0.8186344238975818, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.4215026151661124, + "p10_recall_at_k": 0.8, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9379089615931714, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 558736.0, + "depth": 50, + "doc_recall_at_k": 0.8186344238975818, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.4212259098351151, + "p10_recall_at_k": 0.9, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.984921763869134, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 578336.0, + "depth": 100, + "doc_recall_at_k": 0.8186344238975818, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.42116388796776005, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9899715504978677, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 546569.5021337126, + "depth": 19, + "doc_recall_at_k": 0.8150782361308677, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 18.963015647226175, + "metric": "cosine", + "ndcg_at_k": 0.42000448135267454, + "p10_recall_at_k": 0.8, + "p90_depth": 27, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 1406, + "recall_at_k": 0.9376955903271703, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 553591.7667140824, + "depth": 37, + "doc_recall_at_k": 0.817923186344239, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 36.87695590327169, + "metric": "cosine", + "ndcg_at_k": 0.4209562213288766, + "p10_recall_at_k": 0.9, + "p90_depth": 64, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 1406, + "recall_at_k": 0.9839260312944546, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 568003.7610241821, + "depth": 74, + "doc_recall_at_k": 0.817923186344239, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 73.64224751066857, + "metric": "cosine", + "ndcg_at_k": 0.42094978413016126, + "p10_recall_at_k": 1.0, + "p90_depth": 147, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 1406, + "recall_at_k": 0.9907539118065447, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 569856.0, + "depth": 20, + "doc_recall_at_k": 0.817923186344239, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.42150123906103604, + "p10_recall_at_k": 0.8, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9422475106685625, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 615936.0, + "depth": 50, + "doc_recall_at_k": 0.8193456614509246, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.4215915211492788, + "p10_recall_at_k": 1.0, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9928876244665723, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 692736.0, + "depth": 100, + "doc_recall_at_k": 0.8193456614509246, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.4215397347670436, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9985064011379802, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 568263.1920341394, + "depth": 19, + "doc_recall_at_k": 0.8150782361308677, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 18.963015647226175, + "metric": "cosine", + "ndcg_at_k": 0.4201803385248305, + "p10_recall_at_k": 0.8, + "p90_depth": 27, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 1406, + "recall_at_k": 0.9416073968705555, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 595779.0042674253, + "depth": 37, + "doc_recall_at_k": 0.8193456614509246, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 36.87695590327169, + "metric": "cosine", + "ndcg_at_k": 0.4215409978238593, + "p10_recall_at_k": 1.0, + "p90_depth": 64, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 1406, + "recall_at_k": 0.991465149359887, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 652250.4921763869, + "depth": 74, + "doc_recall_at_k": 0.8186344238975818, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 73.64224751066857, + "metric": "cosine", + "ndcg_at_k": 0.4213256309294448, + "p10_recall_at_k": 1.0, + "p90_depth": 147, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 1406, + "recall_at_k": 0.9995021337126602, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 539136.0, + "depth": 0, + "doc_recall_at_k": 0.7923186344238976, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.40665845499575964, + "p10_recall_at_k": 0.7, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 1406, + "recall_at_k": 0.816998577524889, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.9928876244665719, + "vectors": 11232 + }, + { + "bytes_read_per_query": 546976.0, + "depth": 20, + "doc_recall_at_k": 0.8193456614509246, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.4214058373055683, + "p10_recall_at_k": 0.9, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9668563300142251, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 558736.0, + "depth": 50, + "doc_recall_at_k": 0.817923186344239, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.42094978413016126, + "p10_recall_at_k": 0.9, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9896159317211959, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 578336.0, + "depth": 100, + "doc_recall_at_k": 0.817923186344239, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.42094978413016126, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9910384068278817, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 545257.1664295875, + "depth": 16, + "doc_recall_at_k": 0.8200568990042674, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 15.615220483641536, + "metric": "cosine", + "ndcg_at_k": 0.4216592381152445, + "p10_recall_at_k": 0.8, + "p90_depth": 21, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 1406, + "recall_at_k": 0.9438833570412531, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 548893.3428165007, + "depth": 25, + "doc_recall_at_k": 0.817923186344239, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 24.891180654338548, + "metric": "cosine", + "ndcg_at_k": 0.4209600507596931, + "p10_recall_at_k": 0.9, + "p90_depth": 40, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 1406, + "recall_at_k": 0.9838549075391195, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 554828.2674253201, + "depth": 40, + "doc_recall_at_k": 0.817923186344239, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 40.03129445234708, + "metric": "cosine", + "ndcg_at_k": 0.42094978413016126, + "p10_recall_at_k": 1.0, + "p90_depth": 73, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 1406, + "recall_at_k": 0.9904694167852078, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 569856.0, + "depth": 20, + "doc_recall_at_k": 0.8193456614509246, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.4215453094091963, + "p10_recall_at_k": 0.9, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9719772403982927, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 615936.0, + "depth": 50, + "doc_recall_at_k": 0.8186344238975818, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.4213256309294448, + "p10_recall_at_k": 1.0, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9983641536273113, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 692736.0, + "depth": 100, + "doc_recall_at_k": 0.8186344238975818, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.4213256309294448, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9999288762446658, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 563120.9786628734, + "depth": 16, + "doc_recall_at_k": 0.8193456614509246, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 15.615220483641536, + "metric": "cosine", + "ndcg_at_k": 0.421559761683985, + "p10_recall_at_k": 0.9, + "p90_depth": 21, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 1406, + "recall_at_k": 0.9472972972972986, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 577368.853485064, + "depth": 25, + "doc_recall_at_k": 0.8186344238975818, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 24.891180654338548, + "metric": "cosine", + "ndcg_at_k": 0.4213358975589766, + "p10_recall_at_k": 1.0, + "p90_depth": 40, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 1406, + "recall_at_k": 0.991251778093884, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 600624.0682788051, + "depth": 40, + "doc_recall_at_k": 0.8186344238975818, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 40.03129445234708, + "metric": "cosine", + "ndcg_at_k": 0.4213256309294448, + "p10_recall_at_k": 1.0, + "p90_depth": 73, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 1406, + "recall_at_k": 0.9992887624466575, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 539136.0, + "depth": 0, + "doc_recall_at_k": 0.802275960170697, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.41120030200634866, + "p10_recall_at_k": 0.7, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 1406, + "recall_at_k": 0.8297297297297258, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.9886201991465149, + "vectors": 11232 + }, + { + "bytes_read_per_query": 546976.0, + "depth": 20, + "doc_recall_at_k": 0.8186344238975818, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.42123519500381446, + "p10_recall_at_k": 0.9, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9711237553342824, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 558736.0, + "depth": 50, + "doc_recall_at_k": 0.817923186344239, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.42094978413016126, + "p10_recall_at_k": 1.0, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9904694167852073, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 578336.0, + "depth": 100, + "doc_recall_at_k": 0.817923186344239, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.42094978413016126, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.991109530583216, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 545195.2716927454, + "depth": 15, + "doc_recall_at_k": 0.8193456614509246, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 15.45732574679943, + "metric": "cosine", + "ndcg_at_k": 0.4214892943438041, + "p10_recall_at_k": 0.9, + "p90_depth": 20, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 1406, + "recall_at_k": 0.9536984352773836, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 548413.7980085348, + "depth": 24, + "doc_recall_at_k": 0.8186344238975818, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 23.667852062588906, + "metric": "cosine", + "ndcg_at_k": 0.42118442122682376, + "p10_recall_at_k": 0.9, + "p90_depth": 36, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 1406, + "recall_at_k": 0.9860597439544824, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 553577.826458037, + "depth": 37, + "doc_recall_at_k": 0.817923186344239, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 36.84139402560455, + "metric": "cosine", + "ndcg_at_k": 0.42094978413016126, + "p10_recall_at_k": 1.0, + "p90_depth": 64, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 1406, + "recall_at_k": 0.9907539118065447, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 569856.0, + "depth": 20, + "doc_recall_at_k": 0.8193456614509246, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.4216582555536112, + "p10_recall_at_k": 0.9, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.97724039829303, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 615936.0, + "depth": 50, + "doc_recall_at_k": 0.8186344238975818, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.4213256309294448, + "p10_recall_at_k": 1.0, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9992887624466572, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 692736.0, + "depth": 100, + "doc_recall_at_k": 0.8186344238975818, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.4213256309294448, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 1.0, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 562878.452347084, + "depth": 15, + "doc_recall_at_k": 0.8200568990042674, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 15.45732574679943, + "metric": "cosine", + "ndcg_at_k": 0.4219226215231327, + "p10_recall_at_k": 0.9, + "p90_depth": 20, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 1406, + "recall_at_k": 0.9573968705547661, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 575489.8207681365, + "depth": 24, + "doc_recall_at_k": 0.8200568990042674, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 23.667852062588906, + "metric": "cosine", + "ndcg_at_k": 0.42176586178593506, + "p10_recall_at_k": 1.0, + "p90_depth": 36, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 1406, + "recall_at_k": 0.9938833570412523, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 595724.3812233286, + "depth": 37, + "doc_recall_at_k": 0.8186344238975818, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 36.84139402560455, + "metric": "cosine", + "ndcg_at_k": 0.4213256309294448, + "p10_recall_at_k": 1.0, + "p90_depth": 64, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 1406, + "recall_at_k": 0.9996443812233285, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 269568.0, + "depth": 0, + "doc_recall_at_k": 0.7638691322901849, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.3772622749522497, + "p10_recall_at_k": 0.5, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 1406, + "recall_at_k": 0.7120910384068273, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.9743954480796586, + "vectors": 11232 + }, + { + "bytes_read_per_query": 277408.0, + "depth": 20, + "doc_recall_at_k": 0.8108108108108109, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.41917841142513224, + "p10_recall_at_k": 0.7, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.8990042674253194, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 289168.0, + "depth": 50, + "doc_recall_at_k": 0.8157894736842105, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.4205337735272126, + "p10_recall_at_k": 0.9, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9735419630156483, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 308768.0, + "depth": 100, + "doc_recall_at_k": 0.817923186344239, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.42099366194480314, + "p10_recall_at_k": 0.9, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9871266002844967, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 278370.43527738267, + "depth": 22, + "doc_recall_at_k": 0.813655761024182, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 22.4551920341394, + "metric": "cosine", + "ndcg_at_k": 0.4197158455248707, + "p10_recall_at_k": 0.8, + "p90_depth": 36, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 1406, + "recall_at_k": 0.9157183499288748, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 289535.186344239, + "depth": 51, + "doc_recall_at_k": 0.8172119487908962, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 50.93669985775249, + "metric": "cosine", + "ndcg_at_k": 0.4207988123837968, + "p10_recall_at_k": 0.9, + "p90_depth": 101, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 1406, + "recall_at_k": 0.978520625889049, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 315248.54623044096, + "depth": 117, + "doc_recall_at_k": 0.8172119487908962, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 116.53200568990043, + "metric": "cosine", + "ndcg_at_k": 0.4207441903703335, + "p10_recall_at_k": 0.9, + "p90_depth": 264, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 1406, + "recall_at_k": 0.9896870554765308, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 300288.0, + "depth": 20, + "doc_recall_at_k": 0.8108108108108109, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.419432075680669, + "p10_recall_at_k": 0.7, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9010668563300135, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 346368.0, + "depth": 50, + "doc_recall_at_k": 0.8172119487908962, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.42113643288141184, + "p10_recall_at_k": 0.9, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9800853485064016, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 423168.0, + "depth": 100, + "doc_recall_at_k": 0.8186344238975818, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.4213822174614036, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9952347083926038, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 304059.17496443813, + "depth": 22, + "doc_recall_at_k": 0.8143669985775249, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 22.4551920341394, + "metric": "cosine", + "ndcg_at_k": 0.4200686981716492, + "p10_recall_at_k": 0.8, + "p90_depth": 36, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 1406, + "recall_at_k": 0.9184210526315778, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 347806.77098150784, + "depth": 51, + "doc_recall_at_k": 0.8186344238975818, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 50.93669985775249, + "metric": "cosine", + "ndcg_at_k": 0.42137849639114733, + "p10_recall_at_k": 0.9, + "p90_depth": 101, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 1406, + "recall_at_k": 0.9854196301564737, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 448561.16073968704, + "depth": 117, + "doc_recall_at_k": 0.8186344238975818, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 116.53200568990043, + "metric": "cosine", + "ndcg_at_k": 0.4213256309294448, + "p10_recall_at_k": 1.0, + "p90_depth": 264, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 1406, + "recall_at_k": 0.9982930298719779, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 269568.0, + "depth": 0, + "doc_recall_at_k": 0.7524893314366998, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.3797346936825903, + "p10_recall_at_k": 0.6, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 1406, + "recall_at_k": 0.7382645803698422, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.984352773826458, + "vectors": 11232 + }, + { + "bytes_read_per_query": 277408.0, + "depth": 20, + "doc_recall_at_k": 0.8065433854907539, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.41664139041929543, + "p10_recall_at_k": 0.8, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9153627311522048, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 289168.0, + "depth": 50, + "doc_recall_at_k": 0.8186344238975818, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.4211341750205566, + "p10_recall_at_k": 0.9, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9783072546230464, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 308768.0, + "depth": 100, + "doc_recall_at_k": 0.817923186344239, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.420991219857018, + "p10_recall_at_k": 0.9, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9891891891891903, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 277620.1706970128, + "depth": 21, + "doc_recall_at_k": 0.8108108108108109, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 20.541251778093883, + "metric": "cosine", + "ndcg_at_k": 0.41845696968491686, + "p10_recall_at_k": 0.8, + "p90_depth": 30, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 1406, + "recall_at_k": 0.9257467994310107, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 286307.45945945947, + "depth": 43, + "doc_recall_at_k": 0.8157894736842105, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 42.7027027027027, + "metric": "cosine", + "ndcg_at_k": 0.4202093797564131, + "p10_recall_at_k": 0.9, + "p90_depth": 74, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 1406, + "recall_at_k": 0.981721194879092, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 303870.2304409673, + "depth": 88, + "doc_recall_at_k": 0.817923186344239, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 87.50568990042674, + "metric": "cosine", + "ndcg_at_k": 0.42094978413016126, + "p10_recall_at_k": 1.0, + "p90_depth": 168, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 1406, + "recall_at_k": 0.9905405405405417, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 300288.0, + "depth": 20, + "doc_recall_at_k": 0.8065433854907539, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.4168595346815579, + "p10_recall_at_k": 0.8, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9181365576102419, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 346368.0, + "depth": 50, + "doc_recall_at_k": 0.8186344238975818, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.4213153802255684, + "p10_recall_at_k": 0.9, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9855618776671422, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 423168.0, + "depth": 100, + "doc_recall_at_k": 0.8186344238975818, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.4213670666563016, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9978662873399715, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 301119.3627311522, + "depth": 21, + "doc_recall_at_k": 0.810099573257468, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 20.541251778093883, + "metric": "cosine", + "ndcg_at_k": 0.41840424228205825, + "p10_recall_at_k": 0.8, + "p90_depth": 30, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 1406, + "recall_at_k": 0.9285917496443816, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 335159.35135135136, + "depth": 43, + "doc_recall_at_k": 0.8165007112375533, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 42.7027027027027, + "metric": "cosine", + "ndcg_at_k": 0.42060149529949087, + "p10_recall_at_k": 1.0, + "p90_depth": 74, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 1406, + "recall_at_k": 0.9894736842105278, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 403976.73968705547, + "depth": 88, + "doc_recall_at_k": 0.8186344238975818, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 87.50568990042674, + "metric": "cosine", + "ndcg_at_k": 0.4213256309294448, + "p10_recall_at_k": 1.0, + "p90_depth": 168, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 1406, + "recall_at_k": 0.9994310099573258, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 269568.0, + "depth": 0, + "doc_recall_at_k": 0.7503556187766715, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.37816398556772146, + "p10_recall_at_k": 0.6, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 1406, + "recall_at_k": 0.744310099573256, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.9850640113798008, + "vectors": 11232 + }, + { + "bytes_read_per_query": 277408.0, + "depth": 20, + "doc_recall_at_k": 0.8072546230440967, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.4171985136980058, + "p10_recall_at_k": 0.8, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.921479374110952, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 289168.0, + "depth": 50, + "doc_recall_at_k": 0.8172119487908962, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.42066193116872314, + "p10_recall_at_k": 0.9, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9810099573257487, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 308768.0, + "depth": 100, + "doc_recall_at_k": 0.817923186344239, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.4209660528739555, + "p10_recall_at_k": 0.9, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9898293029871988, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 277764.8705547653, + "depth": 21, + "doc_recall_at_k": 0.810099573257468, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 20.910384068278805, + "metric": "cosine", + "ndcg_at_k": 0.4185915797717954, + "p10_recall_at_k": 0.8, + "p90_depth": 31, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 1406, + "recall_at_k": 0.9344238975817929, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 286455.7837837838, + "depth": 43, + "doc_recall_at_k": 0.8165007112375533, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 43.08108108108108, + "metric": "cosine", + "ndcg_at_k": 0.42043756070159244, + "p10_recall_at_k": 0.9, + "p90_depth": 74, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 1406, + "recall_at_k": 0.9849217638691339, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 303528.69416785205, + "depth": 87, + "doc_recall_at_k": 0.817923186344239, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 86.6344238975818, + "metric": "cosine", + "ndcg_at_k": 0.42094978413016126, + "p10_recall_at_k": 1.0, + "p90_depth": 165, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 1406, + "recall_at_k": 0.9906116642958761, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 300288.0, + "depth": 20, + "doc_recall_at_k": 0.8086770981507824, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.4177350213523941, + "p10_recall_at_k": 0.8, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9248221906116628, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 346368.0, + "depth": 50, + "doc_recall_at_k": 0.8172119487908962, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.42082686762994065, + "p10_recall_at_k": 1.0, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9888335704125187, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 423168.0, + "depth": 100, + "doc_recall_at_k": 0.8186344238975818, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.42133833964676176, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9984352773826458, + "rescore_ms": 1, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 301686.34992887627, + "depth": 21, + "doc_recall_at_k": 0.8115220483641536, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 20.910384068278805, + "metric": "cosine", + "ndcg_at_k": 0.41907975573697814, + "p10_recall_at_k": 0.8, + "p90_depth": 31, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 1406, + "recall_at_k": 0.9376955903271699, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 335740.54054054053, + "depth": 43, + "doc_recall_at_k": 0.8172119487908962, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 43.08108108108108, + "metric": "cosine", + "ndcg_at_k": 0.42080984747439865, + "p10_recall_at_k": 1.0, + "p90_depth": 74, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 1406, + "recall_at_k": 0.9929587482219069, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 402638.47510668566, + "depth": 87, + "doc_recall_at_k": 0.8186344238975818, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 86.6344238975818, + "metric": "cosine", + "ndcg_at_k": 0.4213256309294448, + "p10_recall_at_k": 1.0, + "p90_depth": 165, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 1406, + "recall_at_k": 0.99950213371266, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 1078272.0, + "depth": 0, + "doc_recall_at_k": 0.8229018492176386, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.4216820791937366, + "p10_recall_at_k": 0.8, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 1406, + "recall_at_k": 0.9030583214793728, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.9985775248933144, + "vectors": 11232 + }, + { + "bytes_read_per_query": 1086112.0, + "depth": 20, + "doc_recall_at_k": 0.8172119487908962, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.4207604591141277, + "p10_recall_at_k": 0.9, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.989331436699859, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 1097872.0, + "depth": 50, + "doc_recall_at_k": 0.817923186344239, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.42094978413016126, + "p10_recall_at_k": 1.0, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.991109530583216, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 1117472.0, + "depth": 100, + "doc_recall_at_k": 0.817923186344239, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.42094978413016126, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.991109530583216, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 1083115.960170697, + "depth": 12, + "doc_recall_at_k": 0.8186344238975818, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 12.357041251778094, + "metric": "cosine", + "ndcg_at_k": 0.42122622020393125, + "p10_recall_at_k": 0.9, + "p90_depth": 15, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 1406, + "recall_at_k": 0.9666429587482245, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 1084276.347083926, + "depth": 15, + "doc_recall_at_k": 0.8157894736842105, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 15.317211948790897, + "metric": "cosine", + "ndcg_at_k": 0.42034151292844896, + "p10_recall_at_k": 0.9, + "p90_depth": 21, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 1406, + "recall_at_k": 0.9857752489331453, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 1085771.3001422475, + "depth": 19, + "doc_recall_at_k": 0.817923186344239, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 19.130867709815078, + "metric": "cosine", + "ndcg_at_k": 0.42094978413016126, + "p10_recall_at_k": 1.0, + "p90_depth": 28, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 1406, + "recall_at_k": 0.9905405405405419, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 1108992.0, + "depth": 20, + "doc_recall_at_k": 0.817923186344239, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.4211327458869339, + "p10_recall_at_k": 1.0, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9975817923186345, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 1155072.0, + "depth": 50, + "doc_recall_at_k": 0.8186344238975818, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.4213256309294448, + "p10_recall_at_k": 1.0, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 1.0, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 1231872.0, + "depth": 100, + "doc_recall_at_k": 0.8186344238975818, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.4213256309294448, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 1.0, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 1097252.415362731, + "depth": 12, + "doc_recall_at_k": 0.8200568990042674, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 12.357041251778094, + "metric": "cosine", + "ndcg_at_k": 0.4218118594025884, + "p10_recall_at_k": 0.9, + "p90_depth": 15, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 1406, + "recall_at_k": 0.9705547652916102, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 1101799.2375533427, + "depth": 15, + "doc_recall_at_k": 0.8172119487908962, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 15.317211948790897, + "metric": "cosine", + "ndcg_at_k": 0.42091444340978923, + "p10_recall_at_k": 1.0, + "p90_depth": 21, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 1406, + "recall_at_k": 0.9933143669985784, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 1107657.012802276, + "depth": 19, + "doc_recall_at_k": 0.8186344238975818, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 19.130867709815078, + "metric": "cosine", + "ndcg_at_k": 0.4213256309294448, + "p10_recall_at_k": 1.0, + "p90_depth": 28, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 1406, + "recall_at_k": 0.999217638691323, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 1078272.0, + "depth": 0, + "doc_recall_at_k": 0.8150782361308677, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.4208681525124105, + "p10_recall_at_k": 0.8, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 1406, + "recall_at_k": 0.9088904694167856, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.9964438122332859, + "vectors": 11232 + }, + { + "bytes_read_per_query": 1086112.0, + "depth": 20, + "doc_recall_at_k": 0.817923186344239, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.42094978413016126, + "p10_recall_at_k": 1.0, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9899715504978673, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 1097872.0, + "depth": 50, + "doc_recall_at_k": 0.817923186344239, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.42094978413016126, + "p10_recall_at_k": 1.0, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.991109530583216, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 1117472.0, + "depth": 100, + "doc_recall_at_k": 0.817923186344239, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.42094978413016126, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.991109530583216, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 1083107.5960170696, + "depth": 12, + "doc_recall_at_k": 0.8186344238975818, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 12.33570412517781, + "metric": "cosine", + "ndcg_at_k": 0.421238928921248, + "p10_recall_at_k": 0.9, + "p90_depth": 15, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 1406, + "recall_at_k": 0.9693456614509266, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 1084268.2617354197, + "depth": 15, + "doc_recall_at_k": 0.817923186344239, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 15.296586059743955, + "metric": "cosine", + "ndcg_at_k": 0.42094978413016126, + "p10_recall_at_k": 0.9, + "p90_depth": 20, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 1406, + "recall_at_k": 0.9884068278805136, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 1085725.8549075392, + "depth": 19, + "doc_recall_at_k": 0.817923186344239, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 19.0149359886202, + "metric": "cosine", + "ndcg_at_k": 0.42094978413016126, + "p10_recall_at_k": 1.0, + "p90_depth": 27, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 1406, + "recall_at_k": 0.9911806543385503, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 1108992.0, + "depth": 20, + "doc_recall_at_k": 0.8186344238975818, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.4213256309294448, + "p10_recall_at_k": 1.0, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9982930298719772, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 1155072.0, + "depth": 50, + "doc_recall_at_k": 0.8186344238975818, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.4213256309294448, + "p10_recall_at_k": 1.0, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 1.0, + "rescore_ms": 1, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 1231872.0, + "depth": 100, + "doc_recall_at_k": 0.8186344238975818, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.4213256309294448, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 1.0, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 1097219.641536273, + "depth": 12, + "doc_recall_at_k": 0.8207681365576103, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 12.33570412517781, + "metric": "cosine", + "ndcg_at_k": 0.42204473994748987, + "p10_recall_at_k": 0.9, + "p90_depth": 15, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 1406, + "recall_at_k": 0.9739687055476546, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 1101767.5561877666, + "depth": 15, + "doc_recall_at_k": 0.8186344238975818, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 15.296586059743955, + "metric": "cosine", + "ndcg_at_k": 0.4213256309294448, + "p10_recall_at_k": 1.0, + "p90_depth": 20, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 1406, + "recall_at_k": 0.9960881934566148, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 1107478.9416785205, + "depth": 19, + "doc_recall_at_k": 0.8186344238975818, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 19.0149359886202, + "metric": "cosine", + "ndcg_at_k": 0.4213256309294448, + "p10_recall_at_k": 1.0, + "p90_depth": 27, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 1406, + "recall_at_k": 0.9999288762446658, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + } + ], + "profiles": [ + { + "depth_for_recall": [ + [ + 0.9, + 18 + ], + [ + 0.95, + 28 + ], + [ + 0.99, + 62 + ], + [ + 1.0, + 606 + ] + ], + "first": "binary", + "first_bytes": 48, + "k": 10, + "median_full_depth": 19.0, + "metric": "cosine", + "neighbour_band": 0.023316502571105957, + "queries": 1406, + "rank_noise": 0.018319500828550522, + "recall_at_k": 0.7481507823613063, + "vectors": 11232 + }, + { + "depth_for_recall": [ + [ + 0.9, + 16 + ], + [ + 0.95, + 22 + ], + [ + 0.99, + 45 + ], + [ + 1.0, + 274 + ] + ], + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "median_full_depth": 17.0, + "metric": "cosine", + "neighbour_band": 0.023316502571105957, + "queries": 1406, + "rank_noise": 0.01837565346242917, + "recall_at_k": 0.7726173541962991, + "vectors": 11232 + }, + { + "depth_for_recall": [ + [ + 0.9, + 14 + ], + [ + 0.95, + 17 + ], + [ + 0.99, + 32 + ], + [ + 1.0, + 191 + ] + ], + "first": "pq48", + "first_bytes": 48, + "k": 10, + "median_full_depth": 15.0, + "metric": "cosine", + "neighbour_band": 0.023316502571105957, + "queries": 1406, + "rank_noise": 0.015867100950084683, + "recall_at_k": 0.8109530583214749, + "vectors": 11232 + }, + { + "depth_for_recall": [ + [ + 0.9, + 16 + ], + [ + 0.95, + 22 + ], + [ + 0.99, + 44 + ], + [ + 1.0, + 309 + ] + ], + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "median_full_depth": 17.0, + "metric": "cosine", + "neighbour_band": 0.023316502571105957, + "queries": 1406, + "rank_noise": 0.018468458250404194, + "recall_at_k": 0.7702702702702678, + "vectors": 11232 + }, + { + "depth_for_recall": [ + [ + 0.9, + 13 + ], + [ + 0.95, + 17 + ], + [ + 0.99, + 29 + ], + [ + 1.0, + 111 + ] + ], + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "median_full_depth": 14.0, + "metric": "cosine", + "neighbour_band": 0.023316502571105957, + "queries": 1406, + "rank_noise": 0.014672916822291139, + "recall_at_k": 0.816998577524889, + "vectors": 11232 + }, + { + "depth_for_recall": [ + [ + 0.9, + 13 + ], + [ + 0.95, + 16 + ], + [ + 0.99, + 26 + ], + [ + 1.0, + 85 + ] + ], + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "median_full_depth": 14.0, + "metric": "cosine", + "neighbour_band": 0.023316502571105957, + "queries": 1406, + "rank_noise": 0.015417192604507149, + "recall_at_k": 0.8297297297297258, + "vectors": 11232 + }, + { + "depth_for_recall": [ + [ + 0.9, + 20 + ], + [ + 0.95, + 32 + ], + [ + 0.99, + 72 + ], + [ + 1.0, + 568 + ] + ], + "first": "pq24", + "first_bytes": 24, + "k": 10, + "median_full_depth": 22.0, + "metric": "cosine", + "neighbour_band": 0.023316502571105957, + "queries": 1406, + "rank_noise": 0.027925833599693847, + "recall_at_k": 0.7120910384068273, + "vectors": 11232 + }, + { + "depth_for_recall": [ + [ + 0.9, + 18 + ], + [ + 0.95, + 27 + ], + [ + 0.99, + 59 + ], + [ + 1.0, + 319 + ] + ], + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "median_full_depth": 19.0, + "metric": "cosine", + "neighbour_band": 0.023316502571105957, + "queries": 1406, + "rank_noise": 0.023129456594414597, + "recall_at_k": 0.7382645803698422, + "vectors": 11232 + }, + { + "depth_for_recall": [ + [ + 0.9, + 18 + ], + [ + 0.95, + 25 + ], + [ + 0.99, + 53 + ], + [ + 1.0, + 220 + ] + ], + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "median_full_depth": 19.0, + "metric": "cosine", + "neighbour_band": 0.023316502571105957, + "queries": 1406, + "rank_noise": 0.024587279517376617, + "recall_at_k": 0.744310099573256, + "vectors": 11232 + }, + { + "depth_for_recall": [ + [ + 0.9, + 10 + ], + [ + 0.95, + 12 + ], + [ + 0.99, + 16 + ], + [ + 1.0, + 48 + ] + ], + "first": "pq96", + "first_bytes": 96, + "k": 10, + "median_full_depth": 11.0, + "metric": "cosine", + "neighbour_band": 0.023316502571105957, + "queries": 1406, + "rank_noise": 0.006742915672471994, + "recall_at_k": 0.9030583214793728, + "vectors": 11232 + }, + { + "depth_for_recall": [ + [ + 0.9, + 10 + ], + [ + 0.95, + 12 + ], + [ + 0.99, + 16 + ], + [ + 1.0, + 35 + ] + ], + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "median_full_depth": 11.0, + "metric": "cosine", + "neighbour_band": 0.023316502571105957, + "queries": 1406, + "rank_noise": 0.0071226996009135365, + "recall_at_k": 0.9088904694167856, + "vectors": 11232 + } + ] +} diff --git a/experiments/2026-11-hybrid-adaptive-matrix/results/arguana-minilm-k10.json b/experiments/2026-11-hybrid-adaptive-matrix/results/arguana-minilm-k10.json new file mode 100644 index 0000000..4a29449 --- /dev/null +++ b/experiments/2026-11-hybrid-adaptive-matrix/results/arguana-minilm-k10.json @@ -0,0 +1,3900 @@ +{ + "cells": [ + { + "bytes_read_per_query": 539136.0, + "depth": 0, + "doc_recall_at_k": 0.7041251778093883, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.3394304159589659, + "p10_recall_at_k": 0.7, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 1406, + "recall_at_k": 0.814793741109527, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.9914651493598862, + "vectors": 11232 + }, + { + "bytes_read_per_query": 546976.0, + "depth": 20, + "doc_recall_at_k": 0.732574679943101, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.3581474655994647, + "p10_recall_at_k": 0.9, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.970768136557611, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 558736.0, + "depth": 50, + "doc_recall_at_k": 0.7411095305832148, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.3611649488918465, + "p10_recall_at_k": 1.0, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9965149359886206, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 578336.0, + "depth": 100, + "doc_recall_at_k": 0.7411095305832148, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.3611649488918465, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.998150782361309, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 545395.1749644381, + "depth": 16, + "doc_recall_at_k": 0.7339971550497866, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 15.96728307254623, + "metric": "cosine", + "ndcg_at_k": 0.358593167137908, + "p10_recall_at_k": 0.9, + "p90_depth": 21, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 1406, + "recall_at_k": 0.9546230440967289, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 548979.2147937411, + "depth": 25, + "doc_recall_at_k": 0.7411095305832148, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 25.110241820768138, + "metric": "cosine", + "ndcg_at_k": 0.3611717024178568, + "p10_recall_at_k": 1.0, + "p90_depth": 39, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 1406, + "recall_at_k": 0.9926031294452357, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 554485.6159317212, + "depth": 39, + "doc_recall_at_k": 0.7411095305832148, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 39.15718349928876, + "metric": "cosine", + "ndcg_at_k": 0.3611649488918465, + "p10_recall_at_k": 1.0, + "p90_depth": 70, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 1406, + "recall_at_k": 0.9977951635846377, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 569856.0, + "depth": 20, + "doc_recall_at_k": 0.732574679943101, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.3580681183914817, + "p10_recall_at_k": 0.9, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9716927453769566, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 615936.0, + "depth": 50, + "doc_recall_at_k": 0.7418207681365576, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.36132092431655044, + "p10_recall_at_k": 1.0, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9982930298719773, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 692736.0, + "depth": 100, + "doc_recall_at_k": 0.7418207681365576, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.36132092431655044, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9999288762446658, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 563661.746799431, + "depth": 16, + "doc_recall_at_k": 0.7339971550497866, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 15.96728307254623, + "metric": "cosine", + "ndcg_at_k": 0.3585393501632382, + "p10_recall_at_k": 0.9, + "p90_depth": 21, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 1406, + "recall_at_k": 0.9554765291607402, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 577705.3314366998, + "depth": 25, + "doc_recall_at_k": 0.7411095305832148, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 25.110241820768138, + "metric": "cosine", + "ndcg_at_k": 0.3611135740049618, + "p10_recall_at_k": 1.0, + "p90_depth": 39, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 1406, + "recall_at_k": 0.9940967283072553, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 599281.4338549075, + "depth": 39, + "doc_recall_at_k": 0.7418207681365576, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 39.15718349928876, + "metric": "cosine", + "ndcg_at_k": 0.36132092431655044, + "p10_recall_at_k": 1.0, + "p90_depth": 70, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 1406, + "recall_at_k": 0.9995021337126602, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 539136.0, + "depth": 0, + "doc_recall_at_k": 0.7162162162162162, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.34389024522303585, + "p10_recall_at_k": 0.7, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 1406, + "recall_at_k": 0.8239687055476486, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.9935988620199147, + "vectors": 11232 + }, + { + "bytes_read_per_query": 546976.0, + "depth": 20, + "doc_recall_at_k": 0.7375533428165008, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.35989378316642145, + "p10_recall_at_k": 0.9, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9743243243243254, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 558736.0, + "depth": 50, + "doc_recall_at_k": 0.7411095305832148, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.3611649488918465, + "p10_recall_at_k": 1.0, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9970839260312947, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 578336.0, + "depth": 100, + "doc_recall_at_k": 0.7411095305832148, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.3611649488918465, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9982219061166433, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 545477.1436699858, + "depth": 16, + "doc_recall_at_k": 0.7418207681365576, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 16.176386913229017, + "metric": "cosine", + "ndcg_at_k": 0.3612500651515019, + "p10_recall_at_k": 0.9, + "p90_depth": 22, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 1406, + "recall_at_k": 0.9613086770981533, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 549099.1009957326, + "depth": 25, + "doc_recall_at_k": 0.7418207681365576, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 25.416073968705547, + "metric": "cosine", + "ndcg_at_k": 0.36137054265167434, + "p10_recall_at_k": 1.0, + "p90_depth": 40, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 1406, + "recall_at_k": 0.9946657183499299, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 554811.8179231863, + "depth": 40, + "doc_recall_at_k": 0.7411095305832148, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 39.98933143669986, + "metric": "cosine", + "ndcg_at_k": 0.3611649488918465, + "p10_recall_at_k": 1.0, + "p90_depth": 72, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 1406, + "recall_at_k": 0.998150782361309, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 569856.0, + "depth": 20, + "doc_recall_at_k": 0.7389758179231863, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.3602468422731821, + "p10_recall_at_k": 0.9, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9753200568990052, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 615936.0, + "depth": 50, + "doc_recall_at_k": 0.7418207681365576, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.36132092431655044, + "p10_recall_at_k": 1.0, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9987908961593174, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 692736.0, + "depth": 100, + "doc_recall_at_k": 0.7418207681365576, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.36132092431655044, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 1.0, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 563982.9302987198, + "depth": 16, + "doc_recall_at_k": 0.7411095305832148, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 16.176386913229017, + "metric": "cosine", + "ndcg_at_k": 0.36098214433923315, + "p10_recall_at_k": 0.9, + "p90_depth": 22, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 1406, + "recall_at_k": 0.9623044096728333, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 578175.0896159317, + "depth": 25, + "doc_recall_at_k": 0.7432432432432432, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 25.416073968705547, + "metric": "cosine", + "ndcg_at_k": 0.36173211183620596, + "p10_recall_at_k": 1.0, + "p90_depth": 40, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 1406, + "recall_at_k": 0.9962304409672839, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 600559.613086771, + "depth": 40, + "doc_recall_at_k": 0.7418207681365576, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 39.98933143669986, + "metric": "cosine", + "ndcg_at_k": 0.36132092431655044, + "p10_recall_at_k": 1.0, + "p90_depth": 72, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 1406, + "recall_at_k": 0.9999288762446658, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 539136.0, + "depth": 0, + "doc_recall_at_k": 0.7261735419630156, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.3484476266305802, + "p10_recall_at_k": 0.7, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 1406, + "recall_at_k": 0.8486486486486458, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.9879089615931721, + "vectors": 11232 + }, + { + "bytes_read_per_query": 546976.0, + "depth": 20, + "doc_recall_at_k": 0.7411095305832148, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.36119873128612784, + "p10_recall_at_k": 0.9, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9835704125177823, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 558736.0, + "depth": 50, + "doc_recall_at_k": 0.7411095305832148, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.3611649488918465, + "p10_recall_at_k": 1.0, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9980085348506403, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 578336.0, + "depth": 100, + "doc_recall_at_k": 0.7411095305832148, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.3611649488918465, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9982219061166433, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 544761.7297297297, + "depth": 14, + "doc_recall_at_k": 0.7467994310099573, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 14.35135135135135, + "metric": "cosine", + "ndcg_at_k": 0.36301574028387135, + "p10_recall_at_k": 0.9, + "p90_depth": 19, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 1406, + "recall_at_k": 0.9604551920341424, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 547063.2660028449, + "depth": 20, + "doc_recall_at_k": 0.7403982930298719, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 20.222617354196302, + "metric": "cosine", + "ndcg_at_k": 0.3609858905053448, + "p10_recall_at_k": 1.0, + "p90_depth": 30, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 1406, + "recall_at_k": 0.9912517780938842, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 550183.6529160739, + "depth": 28, + "doc_recall_at_k": 0.7411095305832148, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 28.182788051209105, + "metric": "cosine", + "ndcg_at_k": 0.3611649488918465, + "p10_recall_at_k": 1.0, + "p90_depth": 48, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 1406, + "recall_at_k": 0.9977951635846377, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 569856.0, + "depth": 20, + "doc_recall_at_k": 0.7411095305832148, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.3611576230287749, + "p10_recall_at_k": 0.9, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9849928876244678, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 615936.0, + "depth": 50, + "doc_recall_at_k": 0.7418207681365576, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.36132092431655044, + "p10_recall_at_k": 1.0, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9997866287339972, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 692736.0, + "depth": 100, + "doc_recall_at_k": 0.7418207681365576, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.36132092431655044, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 1.0, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 561179.6756756756, + "depth": 14, + "doc_recall_at_k": 0.7453769559032717, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 14.35135135135135, + "metric": "cosine", + "ndcg_at_k": 0.36257673074497127, + "p10_recall_at_k": 0.9, + "p90_depth": 19, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 1406, + "recall_at_k": 0.9613086770981538, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 570197.9402560455, + "depth": 20, + "doc_recall_at_k": 0.7411095305832148, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 20.222617354196302, + "metric": "cosine", + "ndcg_at_k": 0.36114186593004866, + "p10_recall_at_k": 1.0, + "p90_depth": 30, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 1406, + "recall_at_k": 0.9929587482219067, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 582424.7624466572, + "depth": 28, + "doc_recall_at_k": 0.7418207681365576, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 28.182788051209105, + "metric": "cosine", + "ndcg_at_k": 0.36132092431655044, + "p10_recall_at_k": 1.0, + "p90_depth": 48, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 1406, + "recall_at_k": 0.9995732574679944, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 539136.0, + "depth": 0, + "doc_recall_at_k": 0.7155049786628734, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.34388064043795596, + "p10_recall_at_k": 0.7, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 1406, + "recall_at_k": 0.826884779516355, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.9935988620199147, + "vectors": 11232 + }, + { + "bytes_read_per_query": 546976.0, + "depth": 20, + "doc_recall_at_k": 0.7403982930298719, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.3607472160111311, + "p10_recall_at_k": 0.9, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9757467994310106, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 558736.0, + "depth": 50, + "doc_recall_at_k": 0.7411095305832148, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.3611649488918465, + "p10_recall_at_k": 1.0, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9971550497866287, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 578336.0, + "depth": 100, + "doc_recall_at_k": 0.7411095305832148, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.3611649488918465, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9982219061166433, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 545441.1778093884, + "depth": 16, + "doc_recall_at_k": 0.7453769559032717, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 16.084637268847796, + "metric": "cosine", + "ndcg_at_k": 0.3622278376944417, + "p10_recall_at_k": 0.9, + "p90_depth": 21, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 1406, + "recall_at_k": 0.96209103840683, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 549026.6116642959, + "depth": 25, + "doc_recall_at_k": 0.7418207681365576, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 25.231152204836416, + "metric": "cosine", + "ndcg_at_k": 0.3613620325739033, + "p10_recall_at_k": 1.0, + "p90_depth": 39, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 1406, + "recall_at_k": 0.9945945945945958, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 554584.5917496444, + "depth": 39, + "doc_recall_at_k": 0.7411095305832148, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 39.40967283072546, + "metric": "cosine", + "ndcg_at_k": 0.3611649488918465, + "p10_recall_at_k": 1.0, + "p90_depth": 71, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 1406, + "recall_at_k": 0.998150782361309, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 569856.0, + "depth": 20, + "doc_recall_at_k": 0.7418207681365576, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.36110878519566275, + "p10_recall_at_k": 0.9, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9768136557610249, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 615936.0, + "depth": 50, + "doc_recall_at_k": 0.7418207681365576, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.36132092431655044, + "p10_recall_at_k": 1.0, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9988620199146513, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 692736.0, + "depth": 100, + "doc_recall_at_k": 0.7418207681365576, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.36132092431655044, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 1.0, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 563842.0028449502, + "depth": 16, + "doc_recall_at_k": 0.7446657183499289, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 16.084637268847796, + "metric": "cosine", + "ndcg_at_k": 0.36195991688217305, + "p10_recall_at_k": 0.9, + "p90_depth": 21, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 1406, + "recall_at_k": 0.9630867709815099, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 577891.0497866287, + "depth": 25, + "doc_recall_at_k": 0.7425320056899004, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 25.231152204836416, + "metric": "cosine", + "ndcg_at_k": 0.36151800799860717, + "p10_recall_at_k": 1.0, + "p90_depth": 39, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 1406, + "recall_at_k": 0.9961593172119497, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 599669.2574679944, + "depth": 39, + "doc_recall_at_k": 0.7418207681365576, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 39.40967283072546, + "metric": "cosine", + "ndcg_at_k": 0.36132092431655044, + "p10_recall_at_k": 1.0, + "p90_depth": 71, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 1406, + "recall_at_k": 0.9999288762446658, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 539136.0, + "depth": 0, + "doc_recall_at_k": 0.7197724039829303, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.35195969232856533, + "p10_recall_at_k": 0.7, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 1406, + "recall_at_k": 0.8666429587482172, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.9935988620199147, + "vectors": 11232 + }, + { + "bytes_read_per_query": 546976.0, + "depth": 20, + "doc_recall_at_k": 0.7368421052631579, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.35988883594402454, + "p10_recall_at_k": 1.0, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9901137980085363, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 558736.0, + "depth": 50, + "doc_recall_at_k": 0.7411095305832148, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.3611649488918465, + "p10_recall_at_k": 1.0, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9982219061166433, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 578336.0, + "depth": 100, + "doc_recall_at_k": 0.7411095305832148, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.3611649488918465, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9982219061166433, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 544454.7652916074, + "depth": 14, + "doc_recall_at_k": 0.7339971550497866, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 13.56827880512091, + "metric": "cosine", + "ndcg_at_k": 0.3591632966673281, + "p10_recall_at_k": 0.9, + "p90_depth": 17, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 1406, + "recall_at_k": 0.9636557610241828, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 546223.7837837838, + "depth": 18, + "doc_recall_at_k": 0.7389758179231863, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 18.08108108108108, + "metric": "cosine", + "ndcg_at_k": 0.3605625670057173, + "p10_recall_at_k": 1.0, + "p90_depth": 26, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 1406, + "recall_at_k": 0.9923897581792331, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 548524.7624466572, + "depth": 24, + "doc_recall_at_k": 0.7403982930298719, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 23.950924608819346, + "metric": "cosine", + "ndcg_at_k": 0.3609508450542477, + "p10_recall_at_k": 1.0, + "p90_depth": 37, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 1406, + "recall_at_k": 0.9977951635846377, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 569856.0, + "depth": 20, + "doc_recall_at_k": 0.7368421052631579, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.35983070753112956, + "p10_recall_at_k": 1.0, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9916785206258901, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 615936.0, + "depth": 50, + "doc_recall_at_k": 0.7418207681365576, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.36132092431655044, + "p10_recall_at_k": 1.0, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 1.0, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 692736.0, + "depth": 100, + "doc_recall_at_k": 0.7418207681365576, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.36132092431655044, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 1.0, + "rescore_ms": 2, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 559976.8762446657, + "depth": 14, + "doc_recall_at_k": 0.7347083926031295, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 13.56827880512091, + "metric": "cosine", + "ndcg_at_k": 0.35931927209203196, + "p10_recall_at_k": 0.9, + "p90_depth": 17, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 1406, + "recall_at_k": 0.9646514935988626, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 566908.5405405406, + "depth": 18, + "doc_recall_at_k": 0.7389758179231863, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 18.08108108108108, + "metric": "cosine", + "ndcg_at_k": 0.36050443859282233, + "p10_recall_at_k": 1.0, + "p90_depth": 26, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 1406, + "recall_at_k": 0.9940967283072556, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 575924.6201991466, + "depth": 24, + "doc_recall_at_k": 0.7411095305832148, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 23.950924608819346, + "metric": "cosine", + "ndcg_at_k": 0.36110682047895154, + "p10_recall_at_k": 1.0, + "p90_depth": 37, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 1406, + "recall_at_k": 0.9995732574679944, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 539136.0, + "depth": 0, + "doc_recall_at_k": 0.7219061166429588, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.35119312279777615, + "p10_recall_at_k": 0.8, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 1406, + "recall_at_k": 0.8863442389758164, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.9928876244665719, + "vectors": 11232 + }, + { + "bytes_read_per_query": 546976.0, + "depth": 20, + "doc_recall_at_k": 0.7389758179231863, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.36052263737905, + "p10_recall_at_k": 1.0, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9960881934566148, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 558736.0, + "depth": 50, + "doc_recall_at_k": 0.7411095305832148, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.3611649488918465, + "p10_recall_at_k": 1.0, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9982219061166433, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 578336.0, + "depth": 100, + "doc_recall_at_k": 0.7411095305832148, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.3611649488918465, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9982219061166433, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 544336.8307254623, + "depth": 13, + "doc_recall_at_k": 0.7304409672830725, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 13.2674253200569, + "metric": "cosine", + "ndcg_at_k": 0.35800877987608565, + "p10_recall_at_k": 0.9, + "p90_depth": 16, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 1406, + "recall_at_k": 0.9736130867709822, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 545875.8349928876, + "depth": 17, + "doc_recall_at_k": 0.7403982930298719, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 17.193456614509245, + "metric": "cosine", + "ndcg_at_k": 0.3609508450542477, + "p10_recall_at_k": 1.0, + "p90_depth": 24, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 1406, + "recall_at_k": 0.9958748221906122, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 547822.173541963, + "depth": 22, + "doc_recall_at_k": 0.7411095305832148, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 22.15860597439545, + "metric": "cosine", + "ndcg_at_k": 0.3611649488918465, + "p10_recall_at_k": 1.0, + "p90_depth": 33, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 1406, + "recall_at_k": 0.9982219061166433, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 569856.0, + "depth": 20, + "doc_recall_at_k": 0.7396870554765291, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.36067861280375385, + "p10_recall_at_k": 1.0, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9976529160739689, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 615936.0, + "depth": 50, + "doc_recall_at_k": 0.7418207681365576, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.36132092431655044, + "p10_recall_at_k": 1.0, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 1.0, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 692736.0, + "depth": 100, + "doc_recall_at_k": 0.7418207681365576, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.36132092431655044, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 1.0, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 559514.7652916074, + "depth": 13, + "doc_recall_at_k": 0.732574679943101, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 13.2674253200569, + "metric": "cosine", + "ndcg_at_k": 0.3585741862686843, + "p10_recall_at_k": 0.9, + "p90_depth": 16, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 1406, + "recall_at_k": 0.9749644381223335, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 565545.1493598862, + "depth": 17, + "doc_recall_at_k": 0.7411095305832148, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 17.193456614509245, + "metric": "cosine", + "ndcg_at_k": 0.36110682047895154, + "p10_recall_at_k": 1.0, + "p90_depth": 24, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 1406, + "recall_at_k": 0.9975106685633005, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 573171.6187766714, + "depth": 22, + "doc_recall_at_k": 0.7418207681365576, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 22.15860597439545, + "metric": "cosine", + "ndcg_at_k": 0.36132092431655044, + "p10_recall_at_k": 1.0, + "p90_depth": 33, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 1406, + "recall_at_k": 0.9999288762446658, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 269568.0, + "depth": 0, + "doc_recall_at_k": 0.6749644381223329, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.3163081901975043, + "p10_recall_at_k": 0.6, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 1406, + "recall_at_k": 0.7433854907539089, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.9566145092460882, + "vectors": 11232 + }, + { + "bytes_read_per_query": 277408.0, + "depth": 20, + "doc_recall_at_k": 0.7332859174964438, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.35824525041819566, + "p10_recall_at_k": 0.8, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9285206258890477, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9978662873399715, + "vectors": 11232 + }, + { + "bytes_read_per_query": 289168.0, + "depth": 50, + "doc_recall_at_k": 0.7396870554765291, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.36030920976583214, + "p10_recall_at_k": 1.0, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9886913229018499, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 308768.0, + "depth": 100, + "doc_recall_at_k": 0.7411095305832148, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.3611649488918465, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9970839260312945, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 277258.28165007115, + "depth": 20, + "doc_recall_at_k": 0.7297297297297297, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 19.61806543385491, + "metric": "cosine", + "ndcg_at_k": 0.35724949759562014, + "p10_recall_at_k": 0.8, + "p90_depth": 30, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 1406, + "recall_at_k": 0.9300142247510664, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9978662873399715, + "vectors": 11232 + }, + { + "bytes_read_per_query": 283117.3712660028, + "depth": 35, + "doc_recall_at_k": 0.7389758179231863, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 34.5647226173542, + "metric": "cosine", + "ndcg_at_k": 0.36057864639021725, + "p10_recall_at_k": 0.9, + "p90_depth": 64, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 1406, + "recall_at_k": 0.9843527738264595, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 293331.6756756757, + "depth": 61, + "doc_recall_at_k": 0.7403982930298719, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 60.62162162162162, + "metric": "cosine", + "ndcg_at_k": 0.36088980464772524, + "p10_recall_at_k": 1.0, + "p90_depth": 127, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 1406, + "recall_at_k": 0.9958036984352778, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 300288.0, + "depth": 20, + "doc_recall_at_k": 0.7347083926031295, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.35856513342429924, + "p10_recall_at_k": 0.8, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9293029871977248, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9978662873399715, + "vectors": 11232 + }, + { + "bytes_read_per_query": 346368.0, + "depth": 50, + "doc_recall_at_k": 0.7396870554765291, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.36024932480117633, + "p10_recall_at_k": 1.0, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9901849217638696, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 423168.0, + "depth": 100, + "doc_recall_at_k": 0.7418207681365576, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.36132092431655044, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9988620199146513, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 299701.3485064011, + "depth": 20, + "doc_recall_at_k": 0.7304409672830725, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 19.61806543385491, + "metric": "cosine", + "ndcg_at_k": 0.35738931707520905, + "p10_recall_at_k": 0.8, + "p90_depth": 30, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 1406, + "recall_at_k": 0.9304409672830722, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9978662873399715, + "vectors": 11232 + }, + { + "bytes_read_per_query": 322659.41394025605, + "depth": 35, + "doc_recall_at_k": 0.7396870554765291, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 34.5647226173542, + "metric": "cosine", + "ndcg_at_k": 0.3607201565458434, + "p10_recall_at_k": 0.9, + "p90_depth": 64, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 1406, + "recall_at_k": 0.9858463726884793, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 362682.8108108108, + "depth": 61, + "doc_recall_at_k": 0.7411095305832148, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 60.62162162162162, + "metric": "cosine", + "ndcg_at_k": 0.3610457800724291, + "p10_recall_at_k": 1.0, + "p90_depth": 127, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 1406, + "recall_at_k": 0.9974395448079663, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 269568.0, + "depth": 0, + "doc_recall_at_k": 0.6785206258890469, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.32686818684537566, + "p10_recall_at_k": 0.6, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 1406, + "recall_at_k": 0.7822190611664258, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.9836415362731152, + "vectors": 11232 + }, + { + "bytes_read_per_query": 277408.0, + "depth": 20, + "doc_recall_at_k": 0.7311522048364154, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.35811415671377167, + "p10_recall_at_k": 0.8, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9571123755334282, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 289168.0, + "depth": 50, + "doc_recall_at_k": 0.7418207681365576, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.36139176144676227, + "p10_recall_at_k": 1.0, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9950213371266008, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 308768.0, + "depth": 100, + "doc_recall_at_k": 0.7411095305832148, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.3611649488918465, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9980796586059747, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 276400.11948790896, + "depth": 17, + "doc_recall_at_k": 0.7261735419630156, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 17.42887624466572, + "metric": "cosine", + "ndcg_at_k": 0.35675792835573694, + "p10_recall_at_k": 0.8, + "p90_depth": 24, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 1406, + "recall_at_k": 0.9465149359886204, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 281041.667140825, + "depth": 29, + "doc_recall_at_k": 0.7418207681365576, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 29.269559032716927, + "metric": "cosine", + "ndcg_at_k": 0.3614260554861116, + "p10_recall_at_k": 1.0, + "p90_depth": 46, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 1406, + "recall_at_k": 0.9916785206258893, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 288506.9530583215, + "depth": 48, + "doc_recall_at_k": 0.7411095305832148, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 48.31365576102418, + "metric": "cosine", + "ndcg_at_k": 0.3611649488918465, + "p10_recall_at_k": 1.0, + "p90_depth": 86, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 1406, + "recall_at_k": 0.9980085348506403, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 300288.0, + "depth": 20, + "doc_recall_at_k": 0.7311522048364154, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.35808633469452716, + "p10_recall_at_k": 0.8, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9581081081081081, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 346368.0, + "depth": 50, + "doc_recall_at_k": 0.7418207681365576, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.36133363303386734, + "p10_recall_at_k": 1.0, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9966571834992889, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 423168.0, + "depth": 100, + "doc_recall_at_k": 0.7418207681365576, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.36132092431655044, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9998577524893314, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 296338.75391180656, + "depth": 17, + "doc_recall_at_k": 0.7261735419630156, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 17.42887624466572, + "metric": "cosine", + "ndcg_at_k": 0.35673010633649244, + "p10_recall_at_k": 0.8, + "p90_depth": 24, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 1406, + "recall_at_k": 0.9474395448079658, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 314526.0426742532, + "depth": 29, + "doc_recall_at_k": 0.7411095305832148, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 29.269559032716927, + "metric": "cosine", + "ndcg_at_k": 0.36116233331338876, + "p10_recall_at_k": 1.0, + "p90_depth": 46, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 1406, + "recall_at_k": 0.9932432432432434, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 343777.7752489331, + "depth": 48, + "doc_recall_at_k": 0.7418207681365576, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 48.31365576102418, + "metric": "cosine", + "ndcg_at_k": 0.36132092431655044, + "p10_recall_at_k": 1.0, + "p90_depth": 86, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 1406, + "recall_at_k": 0.999786628733997, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 269568.0, + "depth": 0, + "doc_recall_at_k": 0.7005689900426743, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.33697329216270516, + "p10_recall_at_k": 0.7, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 1406, + "recall_at_k": 0.8049075391180612, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.984352773826458, + "vectors": 11232 + }, + { + "bytes_read_per_query": 277408.0, + "depth": 20, + "doc_recall_at_k": 0.7332859174964438, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.3587763234148391, + "p10_recall_at_k": 0.9, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9680654338549082, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 289168.0, + "depth": 50, + "doc_recall_at_k": 0.7411095305832148, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.36119611798917145, + "p10_recall_at_k": 1.0, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9963726884779517, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 308768.0, + "depth": 100, + "doc_recall_at_k": 0.7411095305832148, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.3611649488918465, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.998150782361309, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 276324.0056899004, + "depth": 17, + "doc_recall_at_k": 0.7311522048364154, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 17.234708392603128, + "metric": "cosine", + "ndcg_at_k": 0.3582833465994155, + "p10_recall_at_k": 0.9, + "p90_depth": 23, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 1406, + "recall_at_k": 0.9613086770981525, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 280636.28449502133, + "depth": 28, + "doc_recall_at_k": 0.7411095305832148, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 28.23541963015647, + "metric": "cosine", + "ndcg_at_k": 0.3612003166287174, + "p10_recall_at_k": 1.0, + "p90_depth": 44, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 1406, + "recall_at_k": 0.9941678520625895, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 287293.59317211946, + "depth": 45, + "doc_recall_at_k": 0.7411095305832148, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 45.21834992887624, + "metric": "cosine", + "ndcg_at_k": 0.3611649488918465, + "p10_recall_at_k": 1.0, + "p90_depth": 79, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 1406, + "recall_at_k": 0.9978662873399718, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 300288.0, + "depth": 20, + "doc_recall_at_k": 0.7339971550497866, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.358932298839543, + "p10_recall_at_k": 0.9, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9689900426742537, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 346368.0, + "depth": 50, + "doc_recall_at_k": 0.7418207681365576, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.3613520934138753, + "p10_recall_at_k": 1.0, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9980796586059743, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 423168.0, + "depth": 100, + "doc_recall_at_k": 0.7418207681365576, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.36132092431655044, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9999288762446658, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 296040.5120910384, + "depth": 17, + "doc_recall_at_k": 0.7318634423897582, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 17.234708392603128, + "metric": "cosine", + "ndcg_at_k": 0.3584478321018904, + "p10_recall_at_k": 0.9, + "p90_depth": 23, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 1406, + "recall_at_k": 0.9622332859174982, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 312937.6045519203, + "depth": 28, + "doc_recall_at_k": 0.7418207681365576, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 28.23541963015647, + "metric": "cosine", + "ndcg_at_k": 0.3613562920534212, + "p10_recall_at_k": 1.0, + "p90_depth": 44, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 1406, + "recall_at_k": 0.9958748221906121, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 339023.3854907539, + "depth": 45, + "doc_recall_at_k": 0.7418207681365576, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 45.21834992887624, + "metric": "cosine", + "ndcg_at_k": 0.36132092431655044, + "p10_recall_at_k": 1.0, + "p90_depth": 79, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 1406, + "recall_at_k": 0.9996443812233285, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 1078272.0, + "depth": 0, + "doc_recall_at_k": 0.7304409672830725, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.35495992093510664, + "p10_recall_at_k": 0.8, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 1406, + "recall_at_k": 0.9241109530583209, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.9971550497866287, + "vectors": 11232 + }, + { + "bytes_read_per_query": 1086112.0, + "depth": 20, + "doc_recall_at_k": 0.7418207681365576, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.36137905272944537, + "p10_recall_at_k": 1.0, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9972972972972978, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 1097872.0, + "depth": 50, + "doc_recall_at_k": 0.7411095305832148, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.3611649488918465, + "p10_recall_at_k": 1.0, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9982219061166433, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 1117472.0, + "depth": 100, + "doc_recall_at_k": 0.7411095305832148, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.3611649488918465, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9982219061166433, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 1082895.146514936, + "depth": 12, + "doc_recall_at_k": 0.7411095305832148, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 11.793741109530583, + "metric": "cosine", + "ndcg_at_k": 0.3612091430337833, + "p10_recall_at_k": 0.9, + "p90_depth": 14, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 1406, + "recall_at_k": 0.9776671408250373, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 1083698.9416785205, + "depth": 14, + "doc_recall_at_k": 0.7418207681365576, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 13.844238975817923, + "metric": "cosine", + "ndcg_at_k": 0.36137905272944537, + "p10_recall_at_k": 1.0, + "p90_depth": 18, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 1406, + "recall_at_k": 0.9936699857752493, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 1084602.5490753911, + "depth": 16, + "doc_recall_at_k": 0.7411095305832148, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 16.149359886201992, + "metric": "cosine", + "ndcg_at_k": 0.3611649488918465, + "p10_recall_at_k": 1.0, + "p90_depth": 22, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 1406, + "recall_at_k": 0.9975106685633005, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 1108992.0, + "depth": 20, + "doc_recall_at_k": 0.7425320056899004, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.36153502815414923, + "p10_recall_at_k": 1.0, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9990753911806546, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 1155072.0, + "depth": 50, + "doc_recall_at_k": 0.7418207681365576, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.36132092431655044, + "p10_recall_at_k": 1.0, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 1.0, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 1231872.0, + "depth": 100, + "doc_recall_at_k": 0.7418207681365576, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.36132092431655044, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 1.0, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 1096387.186344239, + "depth": 12, + "doc_recall_at_k": 0.7418207681365576, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 11.793741109530583, + "metric": "cosine", + "ndcg_at_k": 0.3613651184584871, + "p10_recall_at_k": 0.9, + "p90_depth": 14, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 1406, + "recall_at_k": 0.9783783783783799, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 1099536.7510668563, + "depth": 14, + "doc_recall_at_k": 0.7425320056899004, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 13.844238975817923, + "metric": "cosine", + "ndcg_at_k": 0.36153502815414923, + "p10_recall_at_k": 1.0, + "p90_depth": 18, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 1406, + "recall_at_k": 0.995448079658606, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 1103077.4167852062, + "depth": 16, + "doc_recall_at_k": 0.7418207681365576, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 16.149359886201992, + "metric": "cosine", + "ndcg_at_k": 0.36132092431655044, + "p10_recall_at_k": 1.0, + "p90_depth": 22, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 1406, + "recall_at_k": 0.9992887624466572, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 1078272.0, + "depth": 0, + "doc_recall_at_k": 0.732574679943101, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.35885015900579137, + "p10_recall_at_k": 0.9, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 1406, + "recall_at_k": 0.9487197724039846, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.9971550497866287, + "vectors": 11232 + }, + { + "bytes_read_per_query": 1086112.0, + "depth": 20, + "doc_recall_at_k": 0.7411095305832148, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.3611649488918465, + "p10_recall_at_k": 1.0, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9982219061166433, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 1097872.0, + "depth": 50, + "doc_recall_at_k": 0.7411095305832148, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.3611649488918465, + "p10_recall_at_k": 1.0, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9982219061166433, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 1117472.0, + "depth": 100, + "doc_recall_at_k": 0.7411095305832148, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.3611649488918465, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 0.9982219061166433, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 1082663.7382645803, + "depth": 11, + "doc_recall_at_k": 0.7382645803698435, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 11.203413940256045, + "metric": "cosine", + "ndcg_at_k": 0.36030677698969044, + "p10_recall_at_k": 0.9, + "p90_depth": 13, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 1406, + "recall_at_k": 0.9864153627311542, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 1083158.0597439546, + "depth": 12, + "doc_recall_at_k": 0.7418207681365576, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 12.46443812233286, + "metric": "cosine", + "ndcg_at_k": 0.36137054265167434, + "p10_recall_at_k": 1.0, + "p90_depth": 15, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 1406, + "recall_at_k": 0.996372688477952, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 1083694.7596017069, + "depth": 14, + "doc_recall_at_k": 0.7411095305832148, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 13.833570412517782, + "metric": "cosine", + "ndcg_at_k": 0.3611649488918465, + "p10_recall_at_k": 1.0, + "p90_depth": 18, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 1406, + "recall_at_k": 0.998150782361309, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 1108992.0, + "depth": 20, + "doc_recall_at_k": 0.7418207681365576, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.36132092431655044, + "p10_recall_at_k": 1.0, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 1.0, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 1155072.0, + "depth": 50, + "doc_recall_at_k": 0.7418207681365576, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.36132092431655044, + "p10_recall_at_k": 1.0, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 1.0, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 1231872.0, + "depth": 100, + "doc_recall_at_k": 0.7418207681365576, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.36132092431655044, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 1406, + "recall_at_k": 1.0, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 1095480.4438122334, + "depth": 11, + "doc_recall_at_k": 0.7382645803698435, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 11.203413940256045, + "metric": "cosine", + "ndcg_at_k": 0.3602571586545665, + "p10_recall_at_k": 0.9, + "p90_depth": 13, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 1406, + "recall_at_k": 0.9876955903271711, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 1097417.3769559034, + "depth": 12, + "doc_recall_at_k": 0.7418207681365576, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 12.46443812233286, + "metric": "cosine", + "ndcg_at_k": 0.3613209243165504, + "p10_recall_at_k": 1.0, + "p90_depth": 15, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 1406, + "recall_at_k": 0.9980085348506403, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + }, + { + "bytes_read_per_query": 1099520.3641536273, + "depth": 14, + "doc_recall_at_k": 0.7418207681365576, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 1406, + "mean_depth": 13.833570412517782, + "metric": "cosine", + "ndcg_at_k": 0.36132092431655044, + "p10_recall_at_k": 1.0, + "p90_depth": 18, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 1406, + "recall_at_k": 0.9999288762446658, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 11232 + } + ], + "profiles": [ + { + "depth_for_recall": [ + [ + 0.9, + 13 + ], + [ + 0.95, + 17 + ], + [ + 0.99, + 28 + ], + [ + 1.0, + 206 + ] + ], + "first": "binary", + "first_bytes": 48, + "k": 10, + "median_full_depth": 14.0, + "metric": "cosine", + "neighbour_band": 0.052856624126434326, + "queries": 1406, + "rank_noise": 0.027135932180800144, + "recall_at_k": 0.814793741109527, + "vectors": 11232 + }, + { + "depth_for_recall": [ + [ + 0.9, + 13 + ], + [ + 0.95, + 16 + ], + [ + 0.99, + 27 + ], + [ + 1.0, + 88 + ] + ], + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "median_full_depth": 14.0, + "metric": "cosine", + "neighbour_band": 0.052856624126434326, + "queries": 1406, + "rank_noise": 0.027019165604858963, + "recall_at_k": 0.8239687055476486, + "vectors": 11232 + }, + { + "depth_for_recall": [ + [ + 0.9, + 12 + ], + [ + 0.95, + 15 + ], + [ + 0.99, + 23 + ], + [ + 1.0, + 82 + ] + ], + "first": "pq48", + "first_bytes": 48, + "k": 10, + "median_full_depth": 13.0, + "metric": "cosine", + "neighbour_band": 0.052856624126434326, + "queries": 1406, + "rank_noise": 0.02566689404324917, + "recall_at_k": 0.8486486486486458, + "vectors": 11232 + }, + { + "depth_for_recall": [ + [ + 0.9, + 13 + ], + [ + 0.95, + 16 + ], + [ + 0.99, + 26 + ], + [ + 1.0, + 81 + ] + ], + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "median_full_depth": 14.0, + "metric": "cosine", + "neighbour_band": 0.052856624126434326, + "queries": 1406, + "rank_noise": 0.026736429238602692, + "recall_at_k": 0.826884779516355, + "vectors": 11232 + }, + { + "depth_for_recall": [ + [ + 0.9, + 11 + ], + [ + 0.95, + 13 + ], + [ + 0.99, + 20 + ], + [ + 1.0, + 45 + ] + ], + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "median_full_depth": 12.0, + "metric": "cosine", + "neighbour_band": 0.052856624126434326, + "queries": 1406, + "rank_noise": 0.019690706587199795, + "recall_at_k": 0.8666429587482172, + "vectors": 11232 + }, + { + "depth_for_recall": [ + [ + 0.9, + 11 + ], + [ + 0.95, + 13 + ], + [ + 0.99, + 17 + ], + [ + 1.0, + 34 + ] + ], + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "median_full_depth": 12.0, + "metric": "cosine", + "neighbour_band": 0.052856624126434326, + "queries": 1406, + "rank_noise": 0.01937586966298112, + "recall_at_k": 0.8863442389758164, + "vectors": 11232 + }, + { + "depth_for_recall": [ + [ + 0.9, + 17 + ], + [ + 0.95, + 24 + ], + [ + 0.99, + 49 + ], + [ + 1.0, + 283 + ] + ], + "first": "pq24", + "first_bytes": 24, + "k": 10, + "median_full_depth": 18.0, + "metric": "cosine", + "neighbour_band": 0.052856624126434326, + "queries": 1406, + "rank_noise": 0.04914630085769442, + "recall_at_k": 0.7433854907539089, + "vectors": 11232 + }, + { + "depth_for_recall": [ + [ + 0.9, + 15 + ], + [ + 0.95, + 19 + ], + [ + 0.99, + 34 + ], + [ + 1.0, + 177 + ] + ], + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "median_full_depth": 16.0, + "metric": "cosine", + "neighbour_band": 0.052856624126434326, + "queries": 1406, + "rank_noise": 0.0340160366947562, + "recall_at_k": 0.7822190611664258, + "vectors": 11232 + }, + { + "depth_for_recall": [ + [ + 0.9, + 14 + ], + [ + 0.95, + 17 + ], + [ + 0.99, + 29 + ], + [ + 1.0, + 107 + ] + ], + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "median_full_depth": 15.0, + "metric": "cosine", + "neighbour_band": 0.052856624126434326, + "queries": 1406, + "rank_noise": 0.03498100032021578, + "recall_at_k": 0.8049075391180612, + "vectors": 11232 + }, + { + "depth_for_recall": [ + [ + 0.9, + 10 + ], + [ + 0.95, + 11 + ], + [ + 0.99, + 14 + ], + [ + 1.0, + 27 + ] + ], + "first": "pq96", + "first_bytes": 96, + "k": 10, + "median_full_depth": 11.0, + "metric": "cosine", + "neighbour_band": 0.052856624126434326, + "queries": 1406, + "rank_noise": 0.010672786260429074, + "recall_at_k": 0.9241109530583209, + "vectors": 11232 + }, + { + "depth_for_recall": [ + [ + 0.9, + 10 + ], + [ + 0.95, + 11 + ], + [ + 0.99, + 12 + ], + [ + 1.0, + 17 + ] + ], + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "median_full_depth": 10.0, + "metric": "cosine", + "neighbour_band": 0.052856624126434326, + "queries": 1406, + "rank_noise": 0.007506839897796312, + "recall_at_k": 0.9487197724039846, + "vectors": 11232 + } + ] +} diff --git a/experiments/2026-11-hybrid-adaptive-matrix/results/config.toml b/experiments/2026-11-hybrid-adaptive-matrix/results/config.toml new file mode 100644 index 0000000..c0cbbfe --- /dev/null +++ b/experiments/2026-11-hybrid-adaptive-matrix/results/config.toml @@ -0,0 +1,17 @@ +db_path = "/Users/liam/dev/hvec/experiments/2026-11-neighbour-spacing/work/hvec.db" +default_chat = "anthropic" +default_embedder = "bge-small" +metric = "cosine" + +[chat.anthropic] +provider = "anthropic" +model = "claude-opus-5-5" +api_key_env = "ANTHROPIC_API_KEY" + +[embed.bge-small] +provider = "local" +model = "BGESmallENV15" + +[embed.minilm] +provider = "local" +model = "AllMiniLML6V2" diff --git a/experiments/2026-11-hybrid-adaptive-matrix/results/cosqa-bge-small-k10.json b/experiments/2026-11-hybrid-adaptive-matrix/results/cosqa-bge-small-k10.json new file mode 100644 index 0000000..2467d7f --- /dev/null +++ b/experiments/2026-11-hybrid-adaptive-matrix/results/cosqa-bge-small-k10.json @@ -0,0 +1,3900 @@ +{ + "cells": [ + { + "bytes_read_per_query": 991104.0, + "depth": 0, + "doc_recall_at_k": 0.474, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.2619419566777617, + "p10_recall_at_k": 0.2, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 500, + "recall_at_k": 0.6598000000000006, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.636, + "vectors": 20648 + }, + { + "bytes_read_per_query": 998944.0, + "depth": 20, + "doc_recall_at_k": 0.488, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.2702669986909615, + "p10_recall_at_k": 0.4, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.831, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.938, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1010704.0, + "depth": 50, + "doc_recall_at_k": 0.496, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.27114672989209054, + "p10_recall_at_k": 0.8, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.9491999999999996, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.986, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1030304.0, + "depth": 100, + "doc_recall_at_k": 0.492, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.26968185001098427, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.9747999999999998, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.994, + "vectors": 20648 + }, + { + "bytes_read_per_query": 999281.904, + "depth": 21, + "doc_recall_at_k": 0.496, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 20.862, + "metric": "cosine", + "ndcg_at_k": 0.27144658044357783, + "p10_recall_at_k": 0.4, + "p90_depth": 35, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 500, + "recall_at_k": 0.8319999999999999, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.934, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1003975.712, + "depth": 33, + "doc_recall_at_k": 0.504, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 32.836, + "metric": "cosine", + "ndcg_at_k": 0.2733439827238947, + "p10_recall_at_k": 0.7, + "p90_depth": 58, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 500, + "recall_at_k": 0.9175999999999994, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.97, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1012027.392, + "depth": 53, + "doc_recall_at_k": 0.492, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 53.376, + "metric": "cosine", + "ndcg_at_k": 0.26954409263948426, + "p10_recall_at_k": 0.9, + "p90_depth": 109, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 500, + "recall_at_k": 0.9631999999999997, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.99, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1021824.0, + "depth": 20, + "doc_recall_at_k": 0.486, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.2697005038212455, + "p10_recall_at_k": 0.4, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.8334, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.94, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1067904.0, + "depth": 50, + "doc_recall_at_k": 0.496, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.2713304202136745, + "p10_recall_at_k": 0.9, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.9565999999999997, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.992, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1144704.0, + "depth": 100, + "doc_recall_at_k": 0.492, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.26986554033256815, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.9833999999999999, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1023148.032, + "depth": 21, + "doc_recall_at_k": 0.496, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 20.862, + "metric": "cosine", + "ndcg_at_k": 0.27148214556518985, + "p10_recall_at_k": 0.4, + "p90_depth": 35, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 500, + "recall_at_k": 0.8336, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.936, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1041540.096, + "depth": 33, + "doc_recall_at_k": 0.504, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 32.836, + "metric": "cosine", + "ndcg_at_k": 0.27359227972081746, + "p10_recall_at_k": 0.7, + "p90_depth": 58, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 500, + "recall_at_k": 0.9221999999999995, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.972, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1073089.536, + "depth": 53, + "doc_recall_at_k": 0.492, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 53.376, + "metric": "cosine", + "ndcg_at_k": 0.26972778296106814, + "p10_recall_at_k": 0.9, + "p90_depth": 109, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 500, + "recall_at_k": 0.9701999999999998, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.996, + "vectors": 20648 + }, + { + "bytes_read_per_query": 991104.0, + "depth": 0, + "doc_recall_at_k": 0.466, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.252044032538105, + "p10_recall_at_k": 0.2, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 500, + "recall_at_k": 0.7252, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.69, + "vectors": 20648 + }, + { + "bytes_read_per_query": 998944.0, + "depth": 20, + "doc_recall_at_k": 0.482, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.2641119752169854, + "p10_recall_at_k": 0.5, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.8637999999999995, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.936, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1010704.0, + "depth": 50, + "doc_recall_at_k": 0.496, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.2708119297851257, + "p10_recall_at_k": 0.9, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.9541999999999995, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.982, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1030304.0, + "depth": 100, + "doc_recall_at_k": 0.488, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.26810372035834845, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.9827999999999999, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.994, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1009244.192, + "depth": 46, + "doc_recall_at_k": 0.492, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 46.276, + "metric": "cosine", + "ndcg_at_k": 0.26934851667765086, + "p10_recall_at_k": 0.9, + "p90_depth": 90, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 500, + "recall_at_k": 0.9699999999999996, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.986, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1055029.008, + "depth": 163, + "doc_recall_at_k": 0.49, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 163.074, + "metric": "cosine", + "ndcg_at_k": 0.26910372035834845, + "p10_recall_at_k": 1.0, + "p90_depth": 331, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 500, + "recall_at_k": 0.9895999999999998, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.994, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1157852.96, + "depth": 425, + "doc_recall_at_k": 0.49, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 425.38, + "metric": "cosine", + "ndcg_at_k": 0.26910372035834845, + "p10_recall_at_k": 1.0, + "p90_depth": 1000, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 500, + "recall_at_k": 0.9895999999999998, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.994, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1021824.0, + "depth": 20, + "doc_recall_at_k": 0.482, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.2644027044863718, + "p10_recall_at_k": 0.5, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.8685999999999995, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.94, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1067904.0, + "depth": 50, + "doc_recall_at_k": 0.496, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.2710602267820483, + "p10_recall_at_k": 0.9, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.9627999999999997, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.988, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1144704.0, + "depth": 100, + "doc_recall_at_k": 0.488, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.2682874106799324, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.9926, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1062183.936, + "depth": 46, + "doc_recall_at_k": 0.492, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 46.276, + "metric": "cosine", + "ndcg_at_k": 0.2696392459470373, + "p10_recall_at_k": 1.0, + "p90_depth": 90, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 500, + "recall_at_k": 0.9793999999999997, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.992, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1241585.664, + "depth": 163, + "doc_recall_at_k": 0.49, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 163.074, + "metric": "cosine", + "ndcg_at_k": 0.2692874106799324, + "p10_recall_at_k": 1.0, + "p90_depth": 331, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 500, + "recall_at_k": 1.0, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1644487.68, + "depth": 425, + "doc_recall_at_k": 0.49, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 425.38, + "metric": "cosine", + "ndcg_at_k": 0.2692874106799324, + "p10_recall_at_k": 1.0, + "p90_depth": 1000, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 500, + "recall_at_k": 1.0, + "rescore_ms": 2, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 20648 + }, + { + "bytes_read_per_query": 991104.0, + "depth": 0, + "doc_recall_at_k": 0.472, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.25522357072945623, + "p10_recall_at_k": 0.3, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 500, + "recall_at_k": 0.7727999999999999, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.76, + "vectors": 20648 + }, + { + "bytes_read_per_query": 998944.0, + "depth": 20, + "doc_recall_at_k": 0.494, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.2697765499060482, + "p10_recall_at_k": 0.6, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.9017999999999995, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.954, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1010704.0, + "depth": 50, + "doc_recall_at_k": 0.494, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.2702839100023122, + "p10_recall_at_k": 0.9, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.9735999999999997, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.988, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1030304.0, + "depth": 100, + "doc_recall_at_k": 0.49, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.26910372035834845, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.9877999999999997, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.994, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1003414.368, + "depth": 31, + "doc_recall_at_k": 0.492, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 31.404, + "metric": "cosine", + "ndcg_at_k": 0.26973376059956666, + "p10_recall_at_k": 0.9, + "p90_depth": 60, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 500, + "recall_at_k": 0.9649999999999996, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.978, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1018843.488, + "depth": 71, + "doc_recall_at_k": 0.492, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 70.764, + "metric": "cosine", + "ndcg_at_k": 0.2697346501119199, + "p10_recall_at_k": 1.0, + "p90_depth": 156, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 500, + "recall_at_k": 0.9875999999999998, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.992, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1055058.016, + "depth": 163, + "doc_recall_at_k": 0.49, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 163.148, + "metric": "cosine", + "ndcg_at_k": 0.26910372035834845, + "p10_recall_at_k": 1.0, + "p90_depth": 380, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 500, + "recall_at_k": 0.9895999999999998, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.994, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1021824.0, + "depth": 20, + "doc_recall_at_k": 0.494, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.2699313704653887, + "p10_recall_at_k": 0.6, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.9069999999999996, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.96, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1067904.0, + "depth": 50, + "doc_recall_at_k": 0.494, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.2704676003238962, + "p10_recall_at_k": 1.0, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.9817999999999997, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.994, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1144704.0, + "depth": 100, + "doc_recall_at_k": 0.49, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.2692874106799324, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.9973999999999998, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1039340.544, + "depth": 31, + "doc_recall_at_k": 0.492, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 31.404, + "metric": "cosine", + "ndcg_at_k": 0.2700244898689531, + "p10_recall_at_k": 0.9, + "p90_depth": 60, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 500, + "recall_at_k": 0.9725999999999997, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.984, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1099797.504, + "depth": 71, + "doc_recall_at_k": 0.492, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 70.764, + "metric": "cosine", + "ndcg_at_k": 0.2699183404335039, + "p10_recall_at_k": 1.0, + "p90_depth": 156, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 500, + "recall_at_k": 0.9975999999999999, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.998, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1241699.328, + "depth": 163, + "doc_recall_at_k": 0.49, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 163.148, + "metric": "cosine", + "ndcg_at_k": 0.2692874106799324, + "p10_recall_at_k": 1.0, + "p90_depth": 380, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 500, + "recall_at_k": 1.0, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 20648 + }, + { + "bytes_read_per_query": 991104.0, + "depth": 0, + "doc_recall_at_k": 0.462, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.25221334990296185, + "p10_recall_at_k": 0.2, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 500, + "recall_at_k": 0.7205999999999998, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.692, + "vectors": 20648 + }, + { + "bytes_read_per_query": 998944.0, + "depth": 20, + "doc_recall_at_k": 0.486, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.26629895294595457, + "p10_recall_at_k": 0.5, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.8549999999999996, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.932, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1010704.0, + "depth": 50, + "doc_recall_at_k": 0.494, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.27003911368300976, + "p10_recall_at_k": 0.9, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.9523999999999995, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.98, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1030304.0, + "depth": 100, + "doc_recall_at_k": 0.488, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.26810372035834845, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.9827999999999999, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.992, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1009535.84, + "depth": 47, + "doc_recall_at_k": 0.492, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 47.02, + "metric": "cosine", + "ndcg_at_k": 0.2692839100023122, + "p10_recall_at_k": 0.9, + "p90_depth": 92, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 500, + "recall_at_k": 0.9713999999999996, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.988, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1056637.776, + "depth": 167, + "doc_recall_at_k": 0.49, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 167.178, + "metric": "cosine", + "ndcg_at_k": 0.26910372035834845, + "p10_recall_at_k": 1.0, + "p90_depth": 370, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 500, + "recall_at_k": 0.9895999999999998, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.994, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1160658.896, + "depth": 433, + "doc_recall_at_k": 0.49, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 432.538, + "metric": "cosine", + "ndcg_at_k": 0.26910372035834845, + "p10_recall_at_k": 1.0, + "p90_depth": 1000, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 500, + "recall_at_k": 0.9895999999999998, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.994, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1021824.0, + "depth": 20, + "doc_recall_at_k": 0.486, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.2665896822153409, + "p10_recall_at_k": 0.5, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.8595999999999996, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.936, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1067904.0, + "depth": 50, + "doc_recall_at_k": 0.494, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.2702874106799324, + "p10_recall_at_k": 0.9, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.9611999999999996, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.986, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1144704.0, + "depth": 100, + "doc_recall_at_k": 0.488, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.2682874106799324, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.9926, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.998, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1063326.72, + "depth": 47, + "doc_recall_at_k": 0.492, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 47.02, + "metric": "cosine", + "ndcg_at_k": 0.26953220699923486, + "p10_recall_at_k": 1.0, + "p90_depth": 92, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 500, + "recall_at_k": 0.9807999999999999, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.994, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1247889.408, + "depth": 167, + "doc_recall_at_k": 0.49, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 167.178, + "metric": "cosine", + "ndcg_at_k": 0.2692874106799324, + "p10_recall_at_k": 1.0, + "p90_depth": 370, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 500, + "recall_at_k": 1.0, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1655482.368, + "depth": 433, + "doc_recall_at_k": 0.49, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 432.538, + "metric": "cosine", + "ndcg_at_k": 0.2692874106799324, + "p10_recall_at_k": 1.0, + "p90_depth": 1000, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 500, + "recall_at_k": 1.0, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 20648 + }, + { + "bytes_read_per_query": 991104.0, + "depth": 0, + "doc_recall_at_k": 0.482, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.2621567038584593, + "p10_recall_at_k": 0.3, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 500, + "recall_at_k": 0.7641999999999994, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.734, + "vectors": 20648 + }, + { + "bytes_read_per_query": 998944.0, + "depth": 20, + "doc_recall_at_k": 0.488, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.26619573002229907, + "p10_recall_at_k": 0.6, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.9104, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.978, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1010704.0, + "depth": 50, + "doc_recall_at_k": 0.492, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.26968185001098427, + "p10_recall_at_k": 1.0, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.9791999999999998, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.99, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1030304.0, + "depth": 100, + "doc_recall_at_k": 0.49, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.26910372035834845, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.9879999999999999, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.994, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1001810.304, + "depth": 27, + "doc_recall_at_k": 0.49, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 27.312, + "metric": "cosine", + "ndcg_at_k": 0.26890814439651517, + "p10_recall_at_k": 0.9, + "p90_depth": 47, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 500, + "recall_at_k": 0.9603999999999998, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.988, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1013472.304, + "depth": 57, + "doc_recall_at_k": 0.49, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 57.062, + "metric": "cosine", + "ndcg_at_k": 0.26910372035834845, + "p10_recall_at_k": 1.0, + "p90_depth": 112, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 500, + "recall_at_k": 0.9879999999999998, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.994, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1038546.976, + "depth": 121, + "doc_recall_at_k": 0.49, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 121.028, + "metric": "cosine", + "ndcg_at_k": 0.26910372035834845, + "p10_recall_at_k": 1.0, + "p90_depth": 254, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 500, + "recall_at_k": 0.9895999999999998, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.994, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1021824.0, + "depth": 20, + "doc_recall_at_k": 0.486, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.2657484905903116, + "p10_recall_at_k": 0.6, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.9128000000000001, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.98, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1067904.0, + "depth": 50, + "doc_recall_at_k": 0.492, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.26986554033256815, + "p10_recall_at_k": 1.0, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.9885999999999999, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.996, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1144704.0, + "depth": 100, + "doc_recall_at_k": 0.49, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.2692874106799324, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.9984000000000001, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1033055.232, + "depth": 27, + "doc_recall_at_k": 0.49, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 27.312, + "metric": "cosine", + "ndcg_at_k": 0.26909183471809917, + "p10_recall_at_k": 0.9, + "p90_depth": 47, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 500, + "recall_at_k": 0.9685999999999998, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.994, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1078751.232, + "depth": 57, + "doc_recall_at_k": 0.49, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 57.062, + "metric": "cosine", + "ndcg_at_k": 0.2692874106799324, + "p10_recall_at_k": 1.0, + "p90_depth": 112, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 500, + "recall_at_k": 0.9984, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1177003.008, + "depth": 121, + "doc_recall_at_k": 0.49, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 121.028, + "metric": "cosine", + "ndcg_at_k": 0.2692874106799324, + "p10_recall_at_k": 1.0, + "p90_depth": 254, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 500, + "recall_at_k": 1.0, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 20648 + }, + { + "bytes_read_per_query": 991104.0, + "depth": 0, + "doc_recall_at_k": 0.496, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.2720343275967798, + "p10_recall_at_k": 0.3, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 500, + "recall_at_k": 0.7481999999999993, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.726, + "vectors": 20648 + }, + { + "bytes_read_per_query": 998944.0, + "depth": 20, + "doc_recall_at_k": 0.492, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.2682185607522987, + "p10_recall_at_k": 0.6, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.8985999999999996, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.968, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1010704.0, + "depth": 50, + "doc_recall_at_k": 0.494, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.27031277976455576, + "p10_recall_at_k": 1.0, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.9801999999999998, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.992, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1030304.0, + "depth": 100, + "doc_recall_at_k": 0.49, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.26910372035834845, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.9895999999999998, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.994, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1003874.576, + "depth": 33, + "doc_recall_at_k": 0.492, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 32.578, + "metric": "cosine", + "ndcg_at_k": 0.26902205049516936, + "p10_recall_at_k": 0.9, + "p90_depth": 60, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 500, + "recall_at_k": 0.9703999999999994, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.99, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1022067.296, + "depth": 79, + "doc_recall_at_k": 0.49, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 78.988, + "metric": "cosine", + "ndcg_at_k": 0.26910372035834845, + "p10_recall_at_k": 1.0, + "p90_depth": 155, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 500, + "recall_at_k": 0.9895999999999998, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.994, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1064718.464, + "depth": 188, + "doc_recall_at_k": 0.49, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 187.792, + "metric": "cosine", + "ndcg_at_k": 0.26910372035834845, + "p10_recall_at_k": 1.0, + "p90_depth": 399, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 500, + "recall_at_k": 0.9895999999999998, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.994, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1021824.0, + "depth": 20, + "doc_recall_at_k": 0.49, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.2677713213203113, + "p10_recall_at_k": 0.6, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.9009999999999997, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.972, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1067904.0, + "depth": 50, + "doc_recall_at_k": 0.494, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.2704964700861397, + "p10_recall_at_k": 1.0, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.9896, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.998, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1144704.0, + "depth": 100, + "doc_recall_at_k": 0.49, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.2692874106799324, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.9997999999999999, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1041143.808, + "depth": 33, + "doc_recall_at_k": 0.492, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 32.578, + "metric": "cosine", + "ndcg_at_k": 0.26920574081675325, + "p10_recall_at_k": 1.0, + "p90_depth": 60, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 500, + "recall_at_k": 0.9789999999999996, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.996, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1112429.568, + "depth": 79, + "doc_recall_at_k": 0.49, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 78.988, + "metric": "cosine", + "ndcg_at_k": 0.2692874106799324, + "p10_recall_at_k": 1.0, + "p90_depth": 155, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 500, + "recall_at_k": 0.9996, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1279552.512, + "depth": 188, + "doc_recall_at_k": 0.49, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 187.792, + "metric": "cosine", + "ndcg_at_k": 0.2692874106799324, + "p10_recall_at_k": 1.0, + "p90_depth": 399, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 500, + "recall_at_k": 1.0, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 20648 + }, + { + "bytes_read_per_query": 495552.0, + "depth": 0, + "doc_recall_at_k": 0.462, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 500, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.24068518345340473, + "p10_recall_at_k": 0.1, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 500, + "recall_at_k": 0.6733999999999999, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.604, + "vectors": 20648 + }, + { + "bytes_read_per_query": 503392.0, + "depth": 20, + "doc_recall_at_k": 0.46, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 500, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.2551116057297482, + "p10_recall_at_k": 0.3, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.7922, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.87, + "vectors": 20648 + }, + { + "bytes_read_per_query": 515152.0, + "depth": 50, + "doc_recall_at_k": 0.482, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 500, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.26439293603700714, + "p10_recall_at_k": 0.7, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.9195999999999999, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.95, + "vectors": 20648 + }, + { + "bytes_read_per_query": 534752.0, + "depth": 100, + "doc_recall_at_k": 0.486, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 500, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.26786579118989773, + "p10_recall_at_k": 0.9, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.9639999999999994, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.982, + "vectors": 20648 + }, + { + "bytes_read_per_query": 535075.008, + "depth": 101, + "doc_recall_at_k": 0.484, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 500, + "mean_depth": 100.824, + "metric": "cosine", + "ndcg_at_k": 0.2657587522420953, + "p10_recall_at_k": 0.9, + "p90_depth": 214, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 500, + "recall_at_k": 0.9717999999999998, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.984, + "vectors": 20648 + }, + { + "bytes_read_per_query": 668449.088, + "depth": 441, + "doc_recall_at_k": 0.49, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 500, + "mean_depth": 441.064, + "metric": "cosine", + "ndcg_at_k": 0.26910372035834845, + "p10_recall_at_k": 1.0, + "p90_depth": 1000, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 500, + "recall_at_k": 0.9895999999999998, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.994, + "vectors": 20648 + }, + { + "bytes_read_per_query": 809160.6240000001, + "depth": 800, + "doc_recall_at_k": 0.49, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 500, + "mean_depth": 800.022, + "metric": "cosine", + "ndcg_at_k": 0.26910372035834845, + "p10_recall_at_k": 1.0, + "p90_depth": 1000, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 500, + "recall_at_k": 0.9895999999999998, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.994, + "vectors": 20648 + }, + { + "bytes_read_per_query": 526272.0, + "depth": 20, + "doc_recall_at_k": 0.458, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 500, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.2544449390630815, + "p10_recall_at_k": 0.3, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.7964, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.874, + "vectors": 20648 + }, + { + "bytes_read_per_query": 572352.0, + "depth": 50, + "doc_recall_at_k": 0.482, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 500, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.26468366530639353, + "p10_recall_at_k": 0.7, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.9254, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.956, + "vectors": 20648 + }, + { + "bytes_read_per_query": 649152.0, + "depth": 100, + "doc_recall_at_k": 0.486, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 500, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.2680494815114817, + "p10_recall_at_k": 0.9, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.9707999999999996, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.988, + "vectors": 20648 + }, + { + "bytes_read_per_query": 650417.664, + "depth": 101, + "doc_recall_at_k": 0.484, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 500, + "mean_depth": 100.824, + "metric": "cosine", + "ndcg_at_k": 0.2660494815114817, + "p10_recall_at_k": 1.0, + "p90_depth": 214, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 500, + "recall_at_k": 0.9809999999999999, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.99, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1173026.304, + "depth": 441, + "doc_recall_at_k": 0.49, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 500, + "mean_depth": 441.064, + "metric": "cosine", + "ndcg_at_k": 0.2692874106799324, + "p10_recall_at_k": 1.0, + "p90_depth": 1000, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 500, + "recall_at_k": 1.0, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1724385.7920000001, + "depth": 800, + "doc_recall_at_k": 0.49, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 500, + "mean_depth": 800.022, + "metric": "cosine", + "ndcg_at_k": 0.2692874106799324, + "p10_recall_at_k": 1.0, + "p90_depth": 1000, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 500, + "recall_at_k": 1.0, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 20648 + }, + { + "bytes_read_per_query": 495552.0, + "depth": 0, + "doc_recall_at_k": 0.444, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 500, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.2514013666200321, + "p10_recall_at_k": 0.1, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 500, + "recall_at_k": 0.6432000000000001, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.64, + "vectors": 20648 + }, + { + "bytes_read_per_query": 503392.0, + "depth": 20, + "doc_recall_at_k": 0.478, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 500, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.26161714581563733, + "p10_recall_at_k": 0.3, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.7949999999999997, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.914, + "vectors": 20648 + }, + { + "bytes_read_per_query": 515152.0, + "depth": 50, + "doc_recall_at_k": 0.484, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 500, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.26643722548316506, + "p10_recall_at_k": 0.8, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.9345999999999999, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.976, + "vectors": 20648 + }, + { + "bytes_read_per_query": 534752.0, + "depth": 100, + "doc_recall_at_k": 0.492, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 500, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.2690509202574128, + "p10_recall_at_k": 0.9, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.9733999999999997, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.99, + "vectors": 20648 + }, + { + "bytes_read_per_query": 519359.728, + "depth": 61, + "doc_recall_at_k": 0.488, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 500, + "mean_depth": 60.734, + "metric": "cosine", + "ndcg_at_k": 0.26790646752654423, + "p10_recall_at_k": 0.9, + "p90_depth": 122, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 500, + "recall_at_k": 0.9611999999999997, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.986, + "vectors": 20648 + }, + { + "bytes_read_per_query": 593677.44, + "depth": 250, + "doc_recall_at_k": 0.49, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 500, + "mean_depth": 250.32, + "metric": "cosine", + "ndcg_at_k": 0.26910372035834845, + "p10_recall_at_k": 1.0, + "p90_depth": 516, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 500, + "recall_at_k": 0.9891999999999999, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.994, + "vectors": 20648 + }, + { + "bytes_read_per_query": 735993.824, + "depth": 613, + "doc_recall_at_k": 0.49, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 500, + "mean_depth": 613.372, + "metric": "cosine", + "ndcg_at_k": 0.26910372035834845, + "p10_recall_at_k": 1.0, + "p90_depth": 1000, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 500, + "recall_at_k": 0.9895999999999998, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.994, + "vectors": 20648 + }, + { + "bytes_read_per_query": 526272.0, + "depth": 20, + "doc_recall_at_k": 0.478, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 500, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.26230104746162775, + "p10_recall_at_k": 0.3, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.7967999999999996, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.916, + "vectors": 20648 + }, + { + "bytes_read_per_query": 572352.0, + "depth": 50, + "doc_recall_at_k": 0.486, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 500, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.2672229757960769, + "p10_recall_at_k": 0.8, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.9421999999999999, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.982, + "vectors": 20648 + }, + { + "bytes_read_per_query": 649152.0, + "depth": 100, + "doc_recall_at_k": 0.492, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 500, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.2692346105789967, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.9828, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.996, + "vectors": 20648 + }, + { + "bytes_read_per_query": 588839.424, + "depth": 61, + "doc_recall_at_k": 0.488, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 500, + "mean_depth": 60.734, + "metric": "cosine", + "ndcg_at_k": 0.26809015784812823, + "p10_recall_at_k": 0.9, + "p90_depth": 122, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 500, + "recall_at_k": 0.9697999999999999, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.992, + "vectors": 20648 + }, + { + "bytes_read_per_query": 880043.52, + "depth": 250, + "doc_recall_at_k": 0.49, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 500, + "mean_depth": 250.32, + "metric": "cosine", + "ndcg_at_k": 0.2692874106799324, + "p10_recall_at_k": 1.0, + "p90_depth": 516, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 500, + "recall_at_k": 0.9996, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1437691.392, + "depth": 613, + "doc_recall_at_k": 0.49, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 500, + "mean_depth": 613.372, + "metric": "cosine", + "ndcg_at_k": 0.2692874106799324, + "p10_recall_at_k": 1.0, + "p90_depth": 1000, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 500, + "recall_at_k": 1.0, + "rescore_ms": 1, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 20648 + }, + { + "bytes_read_per_query": 495552.0, + "depth": 0, + "doc_recall_at_k": 0.458, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 500, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.24919184858245408, + "p10_recall_at_k": 0.1, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 500, + "recall_at_k": 0.6497999999999998, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.61, + "vectors": 20648 + }, + { + "bytes_read_per_query": 503392.0, + "depth": 20, + "doc_recall_at_k": 0.494, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 500, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.2701613984890901, + "p10_recall_at_k": 0.3, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.8088, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.914, + "vectors": 20648 + }, + { + "bytes_read_per_query": 515152.0, + "depth": 50, + "doc_recall_at_k": 0.484, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 500, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.2653640747870037, + "p10_recall_at_k": 0.7, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.9235999999999999, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.976, + "vectors": 20648 + }, + { + "bytes_read_per_query": 534752.0, + "depth": 100, + "doc_recall_at_k": 0.492, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 500, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.26913240487805734, + "p10_recall_at_k": 0.9, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.9691999999999998, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.986, + "vectors": 20648 + }, + { + "bytes_read_per_query": 524792.848, + "depth": 75, + "doc_recall_at_k": 0.488, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 500, + "mean_depth": 74.594, + "metric": "cosine", + "ndcg_at_k": 0.26790646752654423, + "p10_recall_at_k": 0.9, + "p90_depth": 152, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 500, + "recall_at_k": 0.9662, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.984, + "vectors": 20648 + }, + { + "bytes_read_per_query": 623482.768, + "depth": 326, + "doc_recall_at_k": 0.49, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 500, + "mean_depth": 326.354, + "metric": "cosine", + "ndcg_at_k": 0.26910372035834845, + "p10_recall_at_k": 1.0, + "p90_depth": 722, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 500, + "recall_at_k": 0.9895999999999998, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.994, + "vectors": 20648 + }, + { + "bytes_read_per_query": 773920.608, + "depth": 710, + "doc_recall_at_k": 0.49, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 500, + "mean_depth": 710.124, + "metric": "cosine", + "ndcg_at_k": 0.26910372035834845, + "p10_recall_at_k": 1.0, + "p90_depth": 1000, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 500, + "recall_at_k": 0.9895999999999998, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.994, + "vectors": 20648 + }, + { + "bytes_read_per_query": 526272.0, + "depth": 20, + "doc_recall_at_k": 0.49, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 500, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.2689598448887582, + "p10_recall_at_k": 0.3, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.8101999999999999, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.916, + "vectors": 20648 + }, + { + "bytes_read_per_query": 572352.0, + "depth": 50, + "doc_recall_at_k": 0.484, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 500, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.2655477651085877, + "p10_recall_at_k": 0.7, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.931, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.98, + "vectors": 20648 + }, + { + "bytes_read_per_query": 649152.0, + "depth": 100, + "doc_recall_at_k": 0.492, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 500, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.26931609519964134, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.9783999999999999, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.992, + "vectors": 20648 + }, + { + "bytes_read_per_query": 610128.384, + "depth": 75, + "doc_recall_at_k": 0.488, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 500, + "mean_depth": 74.594, + "metric": "cosine", + "ndcg_at_k": 0.26809015784812823, + "p10_recall_at_k": 1.0, + "p90_depth": 152, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 500, + "recall_at_k": 0.9756, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.99, + "vectors": 20648 + }, + { + "bytes_read_per_query": 996831.744, + "depth": 326, + "doc_recall_at_k": 0.49, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 500, + "mean_depth": 326.354, + "metric": "cosine", + "ndcg_at_k": 0.2692874106799324, + "p10_recall_at_k": 1.0, + "p90_depth": 722, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 500, + "recall_at_k": 1.0, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1586302.4640000002, + "depth": 710, + "doc_recall_at_k": 0.49, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 500, + "mean_depth": 710.124, + "metric": "cosine", + "ndcg_at_k": 0.2692874106799324, + "p10_recall_at_k": 1.0, + "p90_depth": 1000, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 500, + "recall_at_k": 1.0, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1982208.0, + "depth": 0, + "doc_recall_at_k": 0.492, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 500, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.2702221578083903, + "p10_recall_at_k": 0.6, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 500, + "recall_at_k": 0.8775999999999999, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.842, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1990048.0, + "depth": 20, + "doc_recall_at_k": 0.49, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 500, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.26916145988283546, + "p10_recall_at_k": 0.9, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.9691999999999998, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.99, + "vectors": 20648 + }, + { + "bytes_read_per_query": 2001808.0, + "depth": 50, + "doc_recall_at_k": 0.492, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 500, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.2697057803496764, + "p10_recall_at_k": 1.0, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.9885999999999997, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.994, + "vectors": 20648 + }, + { + "bytes_read_per_query": 2021408.0, + "depth": 100, + "doc_recall_at_k": 0.49, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 500, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.26910372035834845, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.9889999999999998, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.994, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1988549.776, + "depth": 16, + "doc_recall_at_k": 0.488, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 500, + "mean_depth": 16.178, + "metric": "cosine", + "ndcg_at_k": 0.2689480383143553, + "p10_recall_at_k": 0.9, + "p90_depth": 25, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 500, + "recall_at_k": 0.9583999999999999, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.982, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1990320.832, + "depth": 21, + "doc_recall_at_k": 0.49, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 500, + "mean_depth": 20.696, + "metric": "cosine", + "ndcg_at_k": 0.26910372035834845, + "p10_recall_at_k": 1.0, + "p90_depth": 34, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 500, + "recall_at_k": 0.9837999999999998, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.992, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1992686.944, + "depth": 27, + "doc_recall_at_k": 0.49, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 500, + "mean_depth": 26.732, + "metric": "cosine", + "ndcg_at_k": 0.26910372035834845, + "p10_recall_at_k": 1.0, + "p90_depth": 46, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 500, + "recall_at_k": 0.9875999999999999, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.994, + "vectors": 20648 + }, + { + "bytes_read_per_query": 2012928.0, + "depth": 20, + "doc_recall_at_k": 0.49, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 500, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.26931628044217587, + "p10_recall_at_k": 1.0, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.9773999999999999, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.996, + "vectors": 20648 + }, + { + "bytes_read_per_query": 2059008.0, + "depth": 50, + "doc_recall_at_k": 0.492, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 500, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.26988947067126035, + "p10_recall_at_k": 1.0, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.999, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 20648 + }, + { + "bytes_read_per_query": 2135808.0, + "depth": 100, + "doc_recall_at_k": 0.49, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 500, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.2692874106799324, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.9994, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 20648 + }, + { + "bytes_read_per_query": 2007057.408, + "depth": 16, + "doc_recall_at_k": 0.49, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 500, + "mean_depth": 16.178, + "metric": "cosine", + "ndcg_at_k": 0.2697337886272672, + "p10_recall_at_k": 0.9, + "p90_depth": 25, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 500, + "recall_at_k": 0.9639999999999999, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.988, + "vectors": 20648 + }, + { + "bytes_read_per_query": 2013997.056, + "depth": 21, + "doc_recall_at_k": 0.49, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 500, + "mean_depth": 20.696, + "metric": "cosine", + "ndcg_at_k": 0.2692874106799324, + "p10_recall_at_k": 1.0, + "p90_depth": 34, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 500, + "recall_at_k": 0.9927999999999999, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.998, + "vectors": 20648 + }, + { + "bytes_read_per_query": 2023268.352, + "depth": 27, + "doc_recall_at_k": 0.49, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 500, + "mean_depth": 26.732, + "metric": "cosine", + "ndcg_at_k": 0.2692874106799324, + "p10_recall_at_k": 1.0, + "p90_depth": 46, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 500, + "recall_at_k": 0.9978, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1982208.0, + "depth": 0, + "doc_recall_at_k": 0.508, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 500, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.2780172817375034, + "p10_recall_at_k": 0.6, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 500, + "recall_at_k": 0.8905999999999993, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.878, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1990048.0, + "depth": 20, + "doc_recall_at_k": 0.494, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 500, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.27031277976455576, + "p10_recall_at_k": 0.9, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.9757999999999998, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.988, + "vectors": 20648 + }, + { + "bytes_read_per_query": 2001808.0, + "depth": 50, + "doc_recall_at_k": 0.49, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 500, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.26910372035834845, + "p10_recall_at_k": 1.0, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.9885999999999998, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.994, + "vectors": 20648 + }, + { + "bytes_read_per_query": 2021408.0, + "depth": 100, + "doc_recall_at_k": 0.49, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 500, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.26910372035834845, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.9895999999999998, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.994, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1989198.928, + "depth": 18, + "doc_recall_at_k": 0.488, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 500, + "mean_depth": 17.834, + "metric": "cosine", + "ndcg_at_k": 0.268472790604777, + "p10_recall_at_k": 0.9, + "p90_depth": 28, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 500, + "recall_at_k": 0.9715999999999996, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.992, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1991678.72, + "depth": 24, + "doc_recall_at_k": 0.49, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 500, + "mean_depth": 24.16, + "metric": "cosine", + "ndcg_at_k": 0.26910372035834845, + "p10_recall_at_k": 1.0, + "p90_depth": 42, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 500, + "recall_at_k": 0.9875999999999997, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.994, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1995419.184, + "depth": 34, + "doc_recall_at_k": 0.49, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 500, + "mean_depth": 33.702, + "metric": "cosine", + "ndcg_at_k": 0.26910372035834845, + "p10_recall_at_k": 1.0, + "p90_depth": 62, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 500, + "recall_at_k": 0.9893999999999997, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.994, + "vectors": 20648 + }, + { + "bytes_read_per_query": 2012928.0, + "depth": 20, + "doc_recall_at_k": 0.492, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 500, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.26986554033256815, + "p10_recall_at_k": 1.0, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.9832, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.994, + "vectors": 20648 + }, + { + "bytes_read_per_query": 2059008.0, + "depth": 50, + "doc_recall_at_k": 0.49, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 500, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.2692874106799324, + "p10_recall_at_k": 1.0, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.999, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 20648 + }, + { + "bytes_read_per_query": 2135808.0, + "depth": 100, + "doc_recall_at_k": 0.49, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 500, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.2692874106799324, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 1.0, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 20648 + }, + { + "bytes_read_per_query": 2009601.024, + "depth": 18, + "doc_recall_at_k": 0.488, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 500, + "mean_depth": 17.834, + "metric": "cosine", + "ndcg_at_k": 0.26865648092636096, + "p10_recall_at_k": 1.0, + "p90_depth": 28, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 500, + "recall_at_k": 0.9797999999999998, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.998, + "vectors": 20648 + }, + { + "bytes_read_per_query": 2019317.76, + "depth": 24, + "doc_recall_at_k": 0.49, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 500, + "mean_depth": 24.16, + "metric": "cosine", + "ndcg_at_k": 0.2692874106799324, + "p10_recall_at_k": 1.0, + "p90_depth": 42, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 500, + "recall_at_k": 0.9975999999999999, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 20648 + }, + { + "bytes_read_per_query": 2033974.2719999999, + "depth": 34, + "doc_recall_at_k": 0.49, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 500, + "mean_depth": 33.702, + "metric": "cosine", + "ndcg_at_k": 0.2692874106799324, + "p10_recall_at_k": 1.0, + "p90_depth": 62, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 500, + "recall_at_k": 0.9997999999999999, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 20648 + } + ], + "profiles": [ + { + "depth_for_recall": [ + [ + 0.9, + 31 + ], + [ + 0.95, + 47 + ], + [ + 0.99, + 132 + ], + [ + 1.0, + 820 + ] + ], + "first": "binary", + "first_bytes": 48, + "k": 10, + "median_full_depth": 18.0, + "metric": "cosine", + "neighbour_band": 0.014807552099227905, + "queries": 500, + "rank_noise": 0.014175589010162073, + "recall_at_k": 0.6598000000000006, + "vectors": 20648 + }, + { + "depth_for_recall": [ + [ + 0.9, + 25 + ], + [ + 0.95, + 42 + ], + [ + 0.99, + 89 + ], + [ + 1.0, + 388 + ] + ], + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "median_full_depth": 15.0, + "metric": "cosine", + "neighbour_band": 0.014807552099227905, + "queries": 500, + "rank_noise": 0.010298020049629607, + "recall_at_k": 0.7252, + "vectors": 20648 + }, + { + "depth_for_recall": [ + [ + 0.9, + 20 + ], + [ + 0.95, + 28 + ], + [ + 0.99, + 68 + ], + [ + 1.0, + 157 + ] + ], + "first": "pq48", + "first_bytes": 48, + "k": 10, + "median_full_depth": 13.0, + "metric": "cosine", + "neighbour_band": 0.014807552099227905, + "queries": 500, + "rank_noise": 0.009104592511394003, + "recall_at_k": 0.7727999999999999, + "vectors": 20648 + }, + { + "depth_for_recall": [ + [ + 0.9, + 27 + ], + [ + 0.95, + 42 + ], + [ + 0.99, + 91 + ], + [ + 1.0, + 472 + ] + ], + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "median_full_depth": 16.0, + "metric": "cosine", + "neighbour_band": 0.014807552099227905, + "queries": 500, + "rank_noise": 0.010485642887275397, + "recall_at_k": 0.7205999999999998, + "vectors": 20648 + }, + { + "depth_for_recall": [ + [ + 0.9, + 19 + ], + [ + 0.95, + 27 + ], + [ + 0.99, + 53 + ], + [ + 1.0, + 111 + ] + ], + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "median_full_depth": 13.0, + "metric": "cosine", + "neighbour_band": 0.014807552099227905, + "queries": 500, + "rank_noise": 0.008935005591655769, + "recall_at_k": 0.7641999999999994, + "vectors": 20648 + }, + { + "depth_for_recall": [ + [ + 0.9, + 20 + ], + [ + 0.95, + 28 + ], + [ + 0.99, + 51 + ], + [ + 1.0, + 104 + ] + ], + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "median_full_depth": 14.0, + "metric": "cosine", + "neighbour_band": 0.014807552099227905, + "queries": 500, + "rank_noise": 0.009199896724115864, + "recall_at_k": 0.7481999999999993, + "vectors": 20648 + }, + { + "depth_for_recall": [ + [ + 0.9, + 38 + ], + [ + 0.95, + 69 + ], + [ + 0.99, + 190 + ], + [ + 1.0, + 705 + ] + ], + "first": "pq24", + "first_bytes": 24, + "k": 10, + "median_full_depth": 23.0, + "metric": "cosine", + "neighbour_band": 0.014807552099227905, + "queries": 500, + "rank_noise": 0.014045970600847607, + "recall_at_k": 0.6733999999999999, + "vectors": 20648 + }, + { + "depth_for_recall": [ + [ + 0.9, + 34 + ], + [ + 0.95, + 55 + ], + [ + 0.99, + 147 + ], + [ + 1.0, + 426 + ] + ], + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "median_full_depth": 20.0, + "metric": "cosine", + "neighbour_band": 0.014807552099227905, + "queries": 500, + "rank_noise": 0.013020915349792024, + "recall_at_k": 0.6432000000000001, + "vectors": 20648 + }, + { + "depth_for_recall": [ + [ + 0.9, + 37 + ], + [ + 0.95, + 63 + ], + [ + 0.99, + 144 + ], + [ + 1.0, + 484 + ] + ], + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "median_full_depth": 20.0, + "metric": "cosine", + "neighbour_band": 0.014807552099227905, + "queries": 500, + "rank_noise": 0.013245933248003865, + "recall_at_k": 0.6497999999999998, + "vectors": 20648 + }, + { + "depth_for_recall": [ + [ + 0.9, + 12 + ], + [ + 0.95, + 16 + ], + [ + 0.99, + 27 + ], + [ + 1.0, + 113 + ] + ], + "first": "pq96", + "first_bytes": 96, + "k": 10, + "median_full_depth": 10.0, + "metric": "cosine", + "neighbour_band": 0.014807552099227905, + "queries": 500, + "rank_noise": 0.004824377917958364, + "recall_at_k": 0.8775999999999999, + "vectors": 20648 + }, + { + "depth_for_recall": [ + [ + 0.9, + 11 + ], + [ + 0.95, + 16 + ], + [ + 0.99, + 24 + ], + [ + 1.0, + 60 + ] + ], + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "median_full_depth": 10.0, + "metric": "cosine", + "neighbour_band": 0.014807552099227905, + "queries": 500, + "rank_noise": 0.004812764609278762, + "recall_at_k": 0.8905999999999993, + "vectors": 20648 + } + ] +} diff --git a/experiments/2026-11-hybrid-adaptive-matrix/results/cosqa-minilm-k10.json b/experiments/2026-11-hybrid-adaptive-matrix/results/cosqa-minilm-k10.json new file mode 100644 index 0000000..f7e3160 --- /dev/null +++ b/experiments/2026-11-hybrid-adaptive-matrix/results/cosqa-minilm-k10.json @@ -0,0 +1,3900 @@ +{ + "cells": [ + { + "bytes_read_per_query": 991104.0, + "depth": 0, + "doc_recall_at_k": 0.472, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.26045612373696797, + "p10_recall_at_k": 0.2, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 500, + "recall_at_k": 0.7190000000000004, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.718, + "vectors": 20648 + }, + { + "bytes_read_per_query": 998944.0, + "depth": 20, + "doc_recall_at_k": 0.498, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.2793901437116567, + "p10_recall_at_k": 0.5, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.8735999999999995, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.966, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1010704.0, + "depth": 50, + "doc_recall_at_k": 0.49, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.2766292580327331, + "p10_recall_at_k": 0.9, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.9691999999999998, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.996, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1030304.0, + "depth": 100, + "doc_recall_at_k": 0.494, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.27783831743894033, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.9910000000000001, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.998, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1002058.048, + "depth": 28, + "doc_recall_at_k": 0.498, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 27.944, + "metric": "cosine", + "ndcg_at_k": 0.27947611434336356, + "p10_recall_at_k": 0.8, + "p90_depth": 49, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 500, + "recall_at_k": 0.9331999999999995, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.988, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1013857.248, + "depth": 58, + "doc_recall_at_k": 0.498, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 58.044, + "metric": "cosine", + "ndcg_at_k": 0.2790424374215963, + "p10_recall_at_k": 1.0, + "p90_depth": 106, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 500, + "recall_at_k": 0.9833999999999999, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.996, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1036728.88, + "depth": 116, + "doc_recall_at_k": 0.498, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 116.39, + "metric": "cosine", + "ndcg_at_k": 0.27915279180448443, + "p10_recall_at_k": 1.0, + "p90_depth": 220, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 500, + "recall_at_k": 0.9954000000000001, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.998, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1021824.0, + "depth": 20, + "doc_recall_at_k": 0.498, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.2793901437116567, + "p10_recall_at_k": 0.5, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.8763999999999995, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.968, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1067904.0, + "depth": 50, + "doc_recall_at_k": 0.49, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.2766292580327331, + "p10_recall_at_k": 1.0, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.9729999999999996, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.998, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1144704.0, + "depth": 100, + "doc_recall_at_k": 0.494, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.27783831743894033, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.9948, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1034025.9839999999, + "depth": 28, + "doc_recall_at_k": 0.498, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 27.944, + "metric": "cosine", + "ndcg_at_k": 0.27947611434336356, + "p10_recall_at_k": 0.8, + "p90_depth": 49, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 500, + "recall_at_k": 0.9367999999999994, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.99, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1080259.584, + "depth": 58, + "doc_recall_at_k": 0.498, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 58.044, + "metric": "cosine", + "ndcg_at_k": 0.2790424374215963, + "p10_recall_at_k": 1.0, + "p90_depth": 106, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 500, + "recall_at_k": 0.9872, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.998, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1169879.04, + "depth": 116, + "doc_recall_at_k": 0.498, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 116.39, + "metric": "cosine", + "ndcg_at_k": 0.27915279180448443, + "p10_recall_at_k": 1.0, + "p90_depth": 220, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 500, + "recall_at_k": 0.9992000000000001, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 20648 + }, + { + "bytes_read_per_query": 991104.0, + "depth": 0, + "doc_recall_at_k": 0.468, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.2542238193998893, + "p10_recall_at_k": 0.2, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 500, + "recall_at_k": 0.7246000000000006, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.71, + "vectors": 20648 + }, + { + "bytes_read_per_query": 998944.0, + "depth": 20, + "doc_recall_at_k": 0.49, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.2754095714284843, + "p10_recall_at_k": 0.6, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.8913999999999994, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.958, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1010704.0, + "depth": 50, + "doc_recall_at_k": 0.498, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.2778243978947023, + "p10_recall_at_k": 0.9, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.9709999999999999, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.988, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1030304.0, + "depth": 100, + "doc_recall_at_k": 0.496, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.2785507318131565, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.9899999999999999, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.998, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1002876.544, + "depth": 30, + "doc_recall_at_k": 0.502, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 30.032, + "metric": "cosine", + "ndcg_at_k": 0.2802444444344172, + "p10_recall_at_k": 0.9, + "p90_depth": 52, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 500, + "recall_at_k": 0.9557999999999995, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.988, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1017654.16, + "depth": 68, + "doc_recall_at_k": 0.496, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 67.73, + "metric": "cosine", + "ndcg_at_k": 0.2785507318131565, + "p10_recall_at_k": 1.0, + "p90_depth": 138, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 500, + "recall_at_k": 0.9920000000000001, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.996, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1048551.6, + "depth": 147, + "doc_recall_at_k": 0.496, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 146.55, + "metric": "cosine", + "ndcg_at_k": 0.2785507318131565, + "p10_recall_at_k": 1.0, + "p90_depth": 306, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 500, + "recall_at_k": 0.9962000000000001, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.998, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1021824.0, + "depth": 20, + "doc_recall_at_k": 0.49, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.2754095714284843, + "p10_recall_at_k": 0.6, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.8933999999999994, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.958, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1067904.0, + "depth": 50, + "doc_recall_at_k": 0.498, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.2778243978947023, + "p10_recall_at_k": 1.0, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.9747999999999999, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.99, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1144704.0, + "depth": 100, + "doc_recall_at_k": 0.496, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.2785507318131565, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.9937999999999998, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1037233.152, + "depth": 30, + "doc_recall_at_k": 0.502, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 30.032, + "metric": "cosine", + "ndcg_at_k": 0.2802444444344172, + "p10_recall_at_k": 0.9, + "p90_depth": 52, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 500, + "recall_at_k": 0.9591999999999994, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.99, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1095137.28, + "depth": 68, + "doc_recall_at_k": 0.496, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 67.73, + "metric": "cosine", + "ndcg_at_k": 0.2785507318131565, + "p10_recall_at_k": 1.0, + "p90_depth": 138, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 500, + "recall_at_k": 0.9958, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.998, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1216204.8, + "depth": 147, + "doc_recall_at_k": 0.496, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 146.55, + "metric": "cosine", + "ndcg_at_k": 0.2785507318131565, + "p10_recall_at_k": 1.0, + "p90_depth": 306, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 500, + "recall_at_k": 1.0, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 20648 + }, + { + "bytes_read_per_query": 991104.0, + "depth": 0, + "doc_recall_at_k": 0.472, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.2586414286624784, + "p10_recall_at_k": 0.3, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 500, + "recall_at_k": 0.775, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.758, + "vectors": 20648 + }, + { + "bytes_read_per_query": 998944.0, + "depth": 20, + "doc_recall_at_k": 0.48, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.2721281668338113, + "p10_recall_at_k": 0.6, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.8955999999999997, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.962, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1010704.0, + "depth": 50, + "doc_recall_at_k": 0.494, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.2778840651464898, + "p10_recall_at_k": 1.0, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.9857999999999996, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.994, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1030304.0, + "depth": 100, + "doc_recall_at_k": 0.496, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.2785507318131565, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.9956, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.998, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1001184.672, + "depth": 26, + "doc_recall_at_k": 0.494, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 25.716, + "metric": "cosine", + "ndcg_at_k": 0.2780666278316806, + "p10_recall_at_k": 0.9, + "p90_depth": 45, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 500, + "recall_at_k": 0.9593999999999997, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.988, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1009557.008, + "depth": 47, + "doc_recall_at_k": 0.496, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 47.074, + "metric": "cosine", + "ndcg_at_k": 0.2785507318131565, + "p10_recall_at_k": 1.0, + "p90_depth": 92, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 500, + "recall_at_k": 0.9906, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.998, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1024007.696, + "depth": 84, + "doc_recall_at_k": 0.496, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 83.938, + "metric": "cosine", + "ndcg_at_k": 0.2785507318131565, + "p10_recall_at_k": 1.0, + "p90_depth": 164, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 500, + "recall_at_k": 0.9948000000000001, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.998, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1021824.0, + "depth": 20, + "doc_recall_at_k": 0.48, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.2721281668338113, + "p10_recall_at_k": 0.6, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.8965999999999997, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.962, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1067904.0, + "depth": 50, + "doc_recall_at_k": 0.494, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.2778840651464898, + "p10_recall_at_k": 1.0, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.9895999999999996, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.996, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1144704.0, + "depth": 100, + "doc_recall_at_k": 0.496, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.2785507318131565, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.9994, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1030603.776, + "depth": 26, + "doc_recall_at_k": 0.494, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 25.716, + "metric": "cosine", + "ndcg_at_k": 0.2780666278316806, + "p10_recall_at_k": 0.9, + "p90_depth": 45, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 500, + "recall_at_k": 0.9629999999999996, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.99, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1063409.6639999999, + "depth": 47, + "doc_recall_at_k": 0.496, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 47.074, + "metric": "cosine", + "ndcg_at_k": 0.2785507318131565, + "p10_recall_at_k": 1.0, + "p90_depth": 92, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 500, + "recall_at_k": 0.9944, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1120032.768, + "depth": 84, + "doc_recall_at_k": 0.496, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 83.938, + "metric": "cosine", + "ndcg_at_k": 0.2785507318131565, + "p10_recall_at_k": 1.0, + "p90_depth": 164, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 500, + "recall_at_k": 0.9986, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 20648 + }, + { + "bytes_read_per_query": 991104.0, + "depth": 0, + "doc_recall_at_k": 0.478, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.2597900381447933, + "p10_recall_at_k": 0.2, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 500, + "recall_at_k": 0.7222000000000008, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.702, + "vectors": 20648 + }, + { + "bytes_read_per_query": 998944.0, + "depth": 20, + "doc_recall_at_k": 0.49, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.275275286706904, + "p10_recall_at_k": 0.6, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.8865999999999995, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.962, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1010704.0, + "depth": 50, + "doc_recall_at_k": 0.498, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.27915773122803567, + "p10_recall_at_k": 0.9, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.9723999999999998, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.99, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1030304.0, + "depth": 100, + "doc_recall_at_k": 0.496, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.2785507318131565, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.9917999999999998, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.996, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1002617.04, + "depth": 29, + "doc_recall_at_k": 0.498, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 29.37, + "metric": "cosine", + "ndcg_at_k": 0.2790872956167925, + "p10_recall_at_k": 0.8, + "p90_depth": 52, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 500, + "recall_at_k": 0.9497999999999996, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.984, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1016775.296, + "depth": 65, + "doc_recall_at_k": 0.496, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 65.488, + "metric": "cosine", + "ndcg_at_k": 0.2785507318131565, + "p10_recall_at_k": 1.0, + "p90_depth": 133, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 500, + "recall_at_k": 0.9946, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.996, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1046097.68, + "depth": 140, + "doc_recall_at_k": 0.496, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 140.29, + "metric": "cosine", + "ndcg_at_k": 0.2785507318131565, + "p10_recall_at_k": 1.0, + "p90_depth": 298, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 500, + "recall_at_k": 0.9962000000000001, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.998, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1021824.0, + "depth": 20, + "doc_recall_at_k": 0.49, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.275275286706904, + "p10_recall_at_k": 0.6, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.8879999999999995, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.962, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1067904.0, + "depth": 50, + "doc_recall_at_k": 0.498, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.27915773122803567, + "p10_recall_at_k": 1.0, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.9761999999999997, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.992, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1144704.0, + "depth": 100, + "doc_recall_at_k": 0.496, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.2785507318131565, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.9955999999999998, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.998, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1036216.32, + "depth": 29, + "doc_recall_at_k": 0.498, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 29.37, + "metric": "cosine", + "ndcg_at_k": 0.2790872956167925, + "p10_recall_at_k": 0.8, + "p90_depth": 52, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 500, + "recall_at_k": 0.9531999999999996, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.986, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1091693.568, + "depth": 65, + "doc_recall_at_k": 0.496, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 65.488, + "metric": "cosine", + "ndcg_at_k": 0.2785507318131565, + "p10_recall_at_k": 1.0, + "p90_depth": 133, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 500, + "recall_at_k": 0.9984, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.998, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1206589.44, + "depth": 140, + "doc_recall_at_k": 0.496, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 140.29, + "metric": "cosine", + "ndcg_at_k": 0.2785507318131565, + "p10_recall_at_k": 1.0, + "p90_depth": 298, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 500, + "recall_at_k": 1.0, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 20648 + }, + { + "bytes_read_per_query": 991104.0, + "depth": 0, + "doc_recall_at_k": 0.488, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.266000513061808, + "p10_recall_at_k": 0.5, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 500, + "recall_at_k": 0.8166, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.794, + "vectors": 20648 + }, + { + "bytes_read_per_query": 998944.0, + "depth": 20, + "doc_recall_at_k": 0.486, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.2751292184998099, + "p10_recall_at_k": 0.8, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.9376, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.986, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1010704.0, + "depth": 50, + "doc_recall_at_k": 0.496, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.2785507318131565, + "p10_recall_at_k": 1.0, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.9896, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.998, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1030304.0, + "depth": 100, + "doc_recall_at_k": 0.496, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.2785507318131565, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.996, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.998, + "vectors": 20648 + }, + { + "bytes_read_per_query": 999426.944, + "depth": 21, + "doc_recall_at_k": 0.49, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 21.232, + "metric": "cosine", + "ndcg_at_k": 0.2763375434299047, + "p10_recall_at_k": 0.9, + "p90_depth": 36, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 500, + "recall_at_k": 0.9613999999999997, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.992, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1004290.88, + "depth": 34, + "doc_recall_at_k": 0.496, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 33.64, + "metric": "cosine", + "ndcg_at_k": 0.2785507318131565, + "p10_recall_at_k": 1.0, + "p90_depth": 64, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 500, + "recall_at_k": 0.9922000000000001, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.998, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1011225.36, + "depth": 51, + "doc_recall_at_k": 0.496, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 51.33, + "metric": "cosine", + "ndcg_at_k": 0.2785507318131565, + "p10_recall_at_k": 1.0, + "p90_depth": 99, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 500, + "recall_at_k": 0.9958, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.998, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1021824.0, + "depth": 20, + "doc_recall_at_k": 0.486, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.2751292184998099, + "p10_recall_at_k": 0.8, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.9394, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.986, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1067904.0, + "depth": 50, + "doc_recall_at_k": 0.496, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.2785507318131565, + "p10_recall_at_k": 1.0, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.9933999999999998, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1144704.0, + "depth": 100, + "doc_recall_at_k": 0.496, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.2785507318131565, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.9997999999999999, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1023716.352, + "depth": 21, + "doc_recall_at_k": 0.49, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 21.232, + "metric": "cosine", + "ndcg_at_k": 0.2763375434299047, + "p10_recall_at_k": 0.9, + "p90_depth": 36, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 500, + "recall_at_k": 0.9637999999999999, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.994, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1042775.04, + "depth": 34, + "doc_recall_at_k": 0.496, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 33.64, + "metric": "cosine", + "ndcg_at_k": 0.2785507318131565, + "p10_recall_at_k": 1.0, + "p90_depth": 64, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 500, + "recall_at_k": 0.9957999999999999, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1069946.88, + "depth": 51, + "doc_recall_at_k": 0.496, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 51.33, + "metric": "cosine", + "ndcg_at_k": 0.2785507318131565, + "p10_recall_at_k": 1.0, + "p90_depth": 99, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 500, + "recall_at_k": 0.9994, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 20648 + }, + { + "bytes_read_per_query": 991104.0, + "depth": 0, + "doc_recall_at_k": 0.504, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.27638478988229503, + "p10_recall_at_k": 0.4, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 500, + "recall_at_k": 0.8169999999999996, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.804, + "vectors": 20648 + }, + { + "bytes_read_per_query": 998944.0, + "depth": 20, + "doc_recall_at_k": 0.498, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.2790760613648565, + "p10_recall_at_k": 0.9, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.9551999999999997, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.992, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1010704.0, + "depth": 50, + "doc_recall_at_k": 0.496, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.2785507318131565, + "p10_recall_at_k": 1.0, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.9925999999999999, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.998, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1030304.0, + "depth": 100, + "doc_recall_at_k": 0.496, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.2785507318131565, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.996, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.998, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1000127.84, + "depth": 23, + "doc_recall_at_k": 0.496, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 23.02, + "metric": "cosine", + "ndcg_at_k": 0.2785507318131565, + "p10_recall_at_k": 1.0, + "p90_depth": 40, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 500, + "recall_at_k": 0.9771999999999997, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.998, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1006172.48, + "depth": 38, + "doc_recall_at_k": 0.496, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 38.44, + "metric": "cosine", + "ndcg_at_k": 0.2785507318131565, + "p10_recall_at_k": 1.0, + "p90_depth": 71, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 500, + "recall_at_k": 0.993, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.998, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1016171.616, + "depth": 64, + "doc_recall_at_k": 0.496, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 63.948, + "metric": "cosine", + "ndcg_at_k": 0.2785507318131565, + "p10_recall_at_k": 1.0, + "p90_depth": 121, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 500, + "recall_at_k": 0.9962000000000001, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.998, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1021824.0, + "depth": 20, + "doc_recall_at_k": 0.498, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.2790760613648565, + "p10_recall_at_k": 0.9, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.9559999999999997, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.992, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1067904.0, + "depth": 50, + "doc_recall_at_k": 0.496, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.2785507318131565, + "p10_recall_at_k": 1.0, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.9964, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1144704.0, + "depth": 100, + "doc_recall_at_k": 0.496, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.2785507318131565, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.9997999999999999, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1026462.72, + "depth": 23, + "doc_recall_at_k": 0.496, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 23.02, + "metric": "cosine", + "ndcg_at_k": 0.2785507318131565, + "p10_recall_at_k": 1.0, + "p90_depth": 40, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 500, + "recall_at_k": 0.9787999999999996, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.998, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1050147.84, + "depth": 38, + "doc_recall_at_k": 0.496, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 38.44, + "metric": "cosine", + "ndcg_at_k": 0.2785507318131565, + "p10_recall_at_k": 1.0, + "p90_depth": 71, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 500, + "recall_at_k": 0.9965999999999999, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1089328.128, + "depth": 64, + "doc_recall_at_k": 0.496, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 500, + "mean_depth": 63.948, + "metric": "cosine", + "ndcg_at_k": 0.2785507318131565, + "p10_recall_at_k": 1.0, + "p90_depth": 121, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 500, + "recall_at_k": 1.0, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 20648 + }, + { + "bytes_read_per_query": 495552.0, + "depth": 0, + "doc_recall_at_k": 0.436, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 500, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.24224725378424777, + "p10_recall_at_k": 0.1, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 500, + "recall_at_k": 0.6399999999999997, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.604, + "vectors": 20648 + }, + { + "bytes_read_per_query": 503392.0, + "depth": 20, + "doc_recall_at_k": 0.484, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 500, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.27171809454861007, + "p10_recall_at_k": 0.3, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.7968000000000002, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.886, + "vectors": 20648 + }, + { + "bytes_read_per_query": 515152.0, + "depth": 50, + "doc_recall_at_k": 0.492, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 500, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.2762354470011055, + "p10_recall_at_k": 0.8, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.9396, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.98, + "vectors": 20648 + }, + { + "bytes_read_per_query": 534752.0, + "depth": 100, + "doc_recall_at_k": 0.492, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 500, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.2759555389726801, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.9792, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.992, + "vectors": 20648 + }, + { + "bytes_read_per_query": 519512.608, + "depth": 61, + "doc_recall_at_k": 0.494, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 500, + "mean_depth": 61.124, + "metric": "cosine", + "ndcg_at_k": 0.27787880853305236, + "p10_recall_at_k": 0.9, + "p90_depth": 115, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 500, + "recall_at_k": 0.9683999999999996, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.984, + "vectors": 20648 + }, + { + "bytes_read_per_query": 574008.448, + "depth": 200, + "doc_recall_at_k": 0.496, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 500, + "mean_depth": 200.144, + "metric": "cosine", + "ndcg_at_k": 0.2785507318131565, + "p10_recall_at_k": 1.0, + "p90_depth": 446, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 500, + "recall_at_k": 0.9957999999999999, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.998, + "vectors": 20648 + }, + { + "bytes_read_per_query": 673097.424, + "depth": 453, + "doc_recall_at_k": 0.496, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 500, + "mean_depth": 452.922, + "metric": "cosine", + "ndcg_at_k": 0.2785507318131565, + "p10_recall_at_k": 1.0, + "p90_depth": 1000, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 500, + "recall_at_k": 0.9962000000000001, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.998, + "vectors": 20648 + }, + { + "bytes_read_per_query": 526272.0, + "depth": 20, + "doc_recall_at_k": 0.484, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 500, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.27171809454861007, + "p10_recall_at_k": 0.3, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.7982000000000001, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.886, + "vectors": 20648 + }, + { + "bytes_read_per_query": 572352.0, + "depth": 50, + "doc_recall_at_k": 0.492, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 500, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.2762354470011055, + "p10_recall_at_k": 0.8, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.9431999999999998, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.982, + "vectors": 20648 + }, + { + "bytes_read_per_query": 649152.0, + "depth": 100, + "doc_recall_at_k": 0.492, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 500, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.2759555389726801, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.9827999999999999, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.994, + "vectors": 20648 + }, + { + "bytes_read_per_query": 589438.464, + "depth": 61, + "doc_recall_at_k": 0.494, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 500, + "mean_depth": 61.124, + "metric": "cosine", + "ndcg_at_k": 0.27787880853305236, + "p10_recall_at_k": 0.9, + "p90_depth": 115, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 500, + "recall_at_k": 0.9719999999999995, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.986, + "vectors": 20648 + }, + { + "bytes_read_per_query": 802973.184, + "depth": 200, + "doc_recall_at_k": 0.496, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 500, + "mean_depth": 200.144, + "metric": "cosine", + "ndcg_at_k": 0.2785507318131565, + "p10_recall_at_k": 1.0, + "p90_depth": 446, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 500, + "recall_at_k": 0.9995999999999999, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1191240.192, + "depth": 453, + "doc_recall_at_k": 0.496, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 500, + "mean_depth": 452.922, + "metric": "cosine", + "ndcg_at_k": 0.2785507318131565, + "p10_recall_at_k": 1.0, + "p90_depth": 1000, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 500, + "recall_at_k": 1.0, + "rescore_ms": 1, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 20648 + }, + { + "bytes_read_per_query": 495552.0, + "depth": 0, + "doc_recall_at_k": 0.476, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 500, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.26002891912026205, + "p10_recall_at_k": 0.2, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 500, + "recall_at_k": 0.6920000000000004, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.65, + "vectors": 20648 + }, + { + "bytes_read_per_query": 503392.0, + "depth": 20, + "doc_recall_at_k": 0.506, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 500, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.27991902188367107, + "p10_recall_at_k": 0.4, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.8429999999999997, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.944, + "vectors": 20648 + }, + { + "bytes_read_per_query": 515152.0, + "depth": 50, + "doc_recall_at_k": 0.498, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 500, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.27915773122803567, + "p10_recall_at_k": 0.9, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.9677999999999999, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.994, + "vectors": 20648 + }, + { + "bytes_read_per_query": 534752.0, + "depth": 100, + "doc_recall_at_k": 0.496, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 500, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.2785507318131565, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.9908, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.998, + "vectors": 20648 + }, + { + "bytes_read_per_query": 510214.368, + "depth": 37, + "doc_recall_at_k": 0.504, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 500, + "mean_depth": 37.404, + "metric": "cosine", + "ndcg_at_k": 0.28105469740848843, + "p10_recall_at_k": 0.9, + "p90_depth": 67, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 500, + "recall_at_k": 0.9545999999999998, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.99, + "vectors": 20648 + }, + { + "bytes_read_per_query": 534693.2, + "depth": 100, + "doc_recall_at_k": 0.496, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 500, + "mean_depth": 99.85, + "metric": "cosine", + "ndcg_at_k": 0.2785507318131565, + "p10_recall_at_k": 1.0, + "p90_depth": 193, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 500, + "recall_at_k": 0.9942000000000001, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.998, + "vectors": 20648 + }, + { + "bytes_read_per_query": 585692.4, + "depth": 230, + "doc_recall_at_k": 0.496, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 500, + "mean_depth": 229.95, + "metric": "cosine", + "ndcg_at_k": 0.2785507318131565, + "p10_recall_at_k": 1.0, + "p90_depth": 460, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 500, + "recall_at_k": 0.9962000000000001, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.998, + "vectors": 20648 + }, + { + "bytes_read_per_query": 526272.0, + "depth": 20, + "doc_recall_at_k": 0.506, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 500, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.27989015212142754, + "p10_recall_at_k": 0.4, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.8435999999999998, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.944, + "vectors": 20648 + }, + { + "bytes_read_per_query": 572352.0, + "depth": 50, + "doc_recall_at_k": 0.498, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 500, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.27915773122803567, + "p10_recall_at_k": 0.9, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.9715999999999998, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.996, + "vectors": 20648 + }, + { + "bytes_read_per_query": 649152.0, + "depth": 100, + "doc_recall_at_k": 0.496, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 500, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.2785507318131565, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.9946, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 20648 + }, + { + "bytes_read_per_query": 553004.544, + "depth": 37, + "doc_recall_at_k": 0.504, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 500, + "mean_depth": 37.404, + "metric": "cosine", + "ndcg_at_k": 0.28105469740848843, + "p10_recall_at_k": 0.9, + "p90_depth": 67, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 500, + "recall_at_k": 0.9581999999999997, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.992, + "vectors": 20648 + }, + { + "bytes_read_per_query": 648921.6, + "depth": 100, + "doc_recall_at_k": 0.496, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 500, + "mean_depth": 99.85, + "metric": "cosine", + "ndcg_at_k": 0.2785507318131565, + "p10_recall_at_k": 1.0, + "p90_depth": 193, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 500, + "recall_at_k": 0.998, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 20648 + }, + { + "bytes_read_per_query": 848755.2, + "depth": 230, + "doc_recall_at_k": 0.496, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 500, + "mean_depth": 229.95, + "metric": "cosine", + "ndcg_at_k": 0.2785507318131565, + "p10_recall_at_k": 1.0, + "p90_depth": 460, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 500, + "recall_at_k": 1.0, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 20648 + }, + { + "bytes_read_per_query": 495552.0, + "depth": 0, + "doc_recall_at_k": 0.474, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 500, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.25524819482783007, + "p10_recall_at_k": 0.2, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 500, + "recall_at_k": 0.6912, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.634, + "vectors": 20648 + }, + { + "bytes_read_per_query": 503392.0, + "depth": 20, + "doc_recall_at_k": 0.498, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 500, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.2790641757246071, + "p10_recall_at_k": 0.5, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.8525999999999998, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.95, + "vectors": 20648 + }, + { + "bytes_read_per_query": 515152.0, + "depth": 50, + "doc_recall_at_k": 0.498, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 500, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.27915773122803567, + "p10_recall_at_k": 0.9, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.9597999999999999, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.996, + "vectors": 20648 + }, + { + "bytes_read_per_query": 534752.0, + "depth": 100, + "doc_recall_at_k": 0.496, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 500, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.2785507318131565, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.9889999999999999, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.998, + "vectors": 20648 + }, + { + "bytes_read_per_query": 513819.2, + "depth": 47, + "doc_recall_at_k": 0.498, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 500, + "mean_depth": 46.6, + "metric": "cosine", + "ndcg_at_k": 0.27919603452345493, + "p10_recall_at_k": 0.9, + "p90_depth": 90, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 500, + "recall_at_k": 0.9691999999999996, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.992, + "vectors": 20648 + }, + { + "bytes_read_per_query": 549615.072, + "depth": 138, + "doc_recall_at_k": 0.496, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 500, + "mean_depth": 137.916, + "metric": "cosine", + "ndcg_at_k": 0.2785507318131565, + "p10_recall_at_k": 1.0, + "p90_depth": 256, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 500, + "recall_at_k": 0.995, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.998, + "vectors": 20648 + }, + { + "bytes_read_per_query": 622074.704, + "depth": 323, + "doc_recall_at_k": 0.496, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 500, + "mean_depth": 322.762, + "metric": "cosine", + "ndcg_at_k": 0.2785507318131565, + "p10_recall_at_k": 1.0, + "p90_depth": 709, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 500, + "recall_at_k": 0.9962000000000001, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.998, + "vectors": 20648 + }, + { + "bytes_read_per_query": 526272.0, + "depth": 20, + "doc_recall_at_k": 0.498, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 500, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.2790641757246071, + "p10_recall_at_k": 0.5, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.8535999999999998, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.95, + "vectors": 20648 + }, + { + "bytes_read_per_query": 572352.0, + "depth": 50, + "doc_recall_at_k": 0.498, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 500, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.27915773122803567, + "p10_recall_at_k": 0.9, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.9618, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.996, + "vectors": 20648 + }, + { + "bytes_read_per_query": 649152.0, + "depth": 100, + "doc_recall_at_k": 0.496, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 500, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.2785507318131565, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.9927999999999998, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 20648 + }, + { + "bytes_read_per_query": 567129.6, + "depth": 47, + "doc_recall_at_k": 0.498, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 500, + "mean_depth": 46.6, + "metric": "cosine", + "ndcg_at_k": 0.27919603452345493, + "p10_recall_at_k": 0.9, + "p90_depth": 90, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 500, + "recall_at_k": 0.9707999999999997, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.992, + "vectors": 20648 + }, + { + "bytes_read_per_query": 707390.976, + "depth": 138, + "doc_recall_at_k": 0.496, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 500, + "mean_depth": 137.916, + "metric": "cosine", + "ndcg_at_k": 0.2785507318131565, + "p10_recall_at_k": 1.0, + "p90_depth": 256, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 500, + "recall_at_k": 0.9987999999999999, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 20648 + }, + { + "bytes_read_per_query": 991314.432, + "depth": 323, + "doc_recall_at_k": 0.496, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 500, + "mean_depth": 322.762, + "metric": "cosine", + "ndcg_at_k": 0.2785507318131565, + "p10_recall_at_k": 1.0, + "p90_depth": 709, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 500, + "recall_at_k": 1.0, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1982208.0, + "depth": 0, + "doc_recall_at_k": 0.488, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 500, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.2713918004616955, + "p10_recall_at_k": 0.7, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 500, + "recall_at_k": 0.8969999999999995, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.892, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1990048.0, + "depth": 20, + "doc_recall_at_k": 0.5, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 500, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.2797922126973732, + "p10_recall_at_k": 0.9, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.9735999999999997, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.996, + "vectors": 20648 + }, + { + "bytes_read_per_query": 2001808.0, + "depth": 50, + "doc_recall_at_k": 0.496, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 500, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.2785507318131565, + "p10_recall_at_k": 1.0, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.996, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.998, + "vectors": 20648 + }, + { + "bytes_read_per_query": 2021408.0, + "depth": 100, + "doc_recall_at_k": 0.496, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 500, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.2785507318131565, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.9962000000000001, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.998, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1988519.2, + "depth": 16, + "doc_recall_at_k": 0.492, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 500, + "mean_depth": 16.1, + "metric": "cosine", + "ndcg_at_k": 0.2768707389169499, + "p10_recall_at_k": 0.9, + "p90_depth": 25, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 500, + "recall_at_k": 0.9675999999999996, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.992, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1989922.56, + "depth": 20, + "doc_recall_at_k": 0.498, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 500, + "mean_depth": 19.68, + "metric": "cosine", + "ndcg_at_k": 0.27912886146579224, + "p10_recall_at_k": 1.0, + "p90_depth": 33, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 500, + "recall_at_k": 0.9877999999999998, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.994, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1991496.832, + "depth": 24, + "doc_recall_at_k": 0.496, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 500, + "mean_depth": 23.696, + "metric": "cosine", + "ndcg_at_k": 0.2785507318131565, + "p10_recall_at_k": 1.0, + "p90_depth": 41, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 500, + "recall_at_k": 0.9944, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.996, + "vectors": 20648 + }, + { + "bytes_read_per_query": 2012928.0, + "depth": 20, + "doc_recall_at_k": 0.5, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 500, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.2797922126973732, + "p10_recall_at_k": 0.9, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.9763999999999996, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.998, + "vectors": 20648 + }, + { + "bytes_read_per_query": 2059008.0, + "depth": 50, + "doc_recall_at_k": 0.496, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 500, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.2785507318131565, + "p10_recall_at_k": 1.0, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.9997999999999999, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 20648 + }, + { + "bytes_read_per_query": 2135808.0, + "depth": 100, + "doc_recall_at_k": 0.496, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 500, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.2785507318131565, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 1.0, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 20648 + }, + { + "bytes_read_per_query": 2006937.6, + "depth": 16, + "doc_recall_at_k": 0.492, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 500, + "mean_depth": 16.1, + "metric": "cosine", + "ndcg_at_k": 0.2768707389169499, + "p10_recall_at_k": 0.9, + "p90_depth": 25, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 500, + "recall_at_k": 0.9703999999999995, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.994, + "vectors": 20648 + }, + { + "bytes_read_per_query": 2012436.48, + "depth": 20, + "doc_recall_at_k": 0.498, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 500, + "mean_depth": 19.68, + "metric": "cosine", + "ndcg_at_k": 0.27912886146579224, + "p10_recall_at_k": 1.0, + "p90_depth": 33, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 500, + "recall_at_k": 0.9913999999999998, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.996, + "vectors": 20648 + }, + { + "bytes_read_per_query": 2018605.056, + "depth": 24, + "doc_recall_at_k": 0.496, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 500, + "mean_depth": 23.696, + "metric": "cosine", + "ndcg_at_k": 0.2785507318131565, + "p10_recall_at_k": 1.0, + "p90_depth": 41, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 500, + "recall_at_k": 0.9982, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.998, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1982208.0, + "depth": 0, + "doc_recall_at_k": 0.5, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 500, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.2762994819113596, + "p10_recall_at_k": 0.8, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 500, + "recall_at_k": 0.9297999999999996, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.908, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1990048.0, + "depth": 20, + "doc_recall_at_k": 0.496, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 500, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.2785507318131565, + "p10_recall_at_k": 1.0, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.9873999999999998, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.998, + "vectors": 20648 + }, + { + "bytes_read_per_query": 2001808.0, + "depth": 50, + "doc_recall_at_k": 0.496, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 500, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.2785507318131565, + "p10_recall_at_k": 1.0, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.9962000000000001, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.998, + "vectors": 20648 + }, + { + "bytes_read_per_query": 2021408.0, + "depth": 100, + "doc_recall_at_k": 0.496, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 500, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.2785507318131565, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.9962000000000001, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.998, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1988315.36, + "depth": 16, + "doc_recall_at_k": 0.5, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 500, + "mean_depth": 15.58, + "metric": "cosine", + "ndcg_at_k": 0.2797837215580559, + "p10_recall_at_k": 1.0, + "p90_depth": 24, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 500, + "recall_at_k": 0.9819999999999998, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.998, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1989517.232, + "depth": 19, + "doc_recall_at_k": 0.498, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 500, + "mean_depth": 18.646, + "metric": "cosine", + "ndcg_at_k": 0.27915279180448443, + "p10_recall_at_k": 1.0, + "p90_depth": 29, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 500, + "recall_at_k": 0.9944, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.998, + "vectors": 20648 + }, + { + "bytes_read_per_query": 1990719.104, + "depth": 22, + "doc_recall_at_k": 0.496, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 500, + "mean_depth": 21.712, + "metric": "cosine", + "ndcg_at_k": 0.2785507318131565, + "p10_recall_at_k": 1.0, + "p90_depth": 36, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 500, + "recall_at_k": 0.9958, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.998, + "vectors": 20648 + }, + { + "bytes_read_per_query": 2012928.0, + "depth": 20, + "doc_recall_at_k": 0.496, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 500, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.2785507318131565, + "p10_recall_at_k": 1.0, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 0.9893999999999998, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.998, + "vectors": 20648 + }, + { + "bytes_read_per_query": 2059008.0, + "depth": 50, + "doc_recall_at_k": 0.496, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 500, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.2785507318131565, + "p10_recall_at_k": 1.0, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 1.0, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 20648 + }, + { + "bytes_read_per_query": 2135808.0, + "depth": 100, + "doc_recall_at_k": 0.496, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 500, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.2785507318131565, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 500, + "recall_at_k": 1.0, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 20648 + }, + { + "bytes_read_per_query": 2006138.88, + "depth": 16, + "doc_recall_at_k": 0.5, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 500, + "mean_depth": 15.58, + "metric": "cosine", + "ndcg_at_k": 0.2797837215580559, + "p10_recall_at_k": 1.0, + "p90_depth": 24, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 500, + "recall_at_k": 0.9855999999999996, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 20648 + }, + { + "bytes_read_per_query": 2010848.256, + "depth": 19, + "doc_recall_at_k": 0.498, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 500, + "mean_depth": 18.646, + "metric": "cosine", + "ndcg_at_k": 0.27915279180448443, + "p10_recall_at_k": 1.0, + "p90_depth": 29, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 500, + "recall_at_k": 0.998, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 20648 + }, + { + "bytes_read_per_query": 2015557.632, + "depth": 22, + "doc_recall_at_k": 0.496, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 500, + "mean_depth": 21.712, + "metric": "cosine", + "ndcg_at_k": 0.2785507318131565, + "p10_recall_at_k": 1.0, + "p90_depth": 36, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 500, + "recall_at_k": 0.9994, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 20648 + } + ], + "profiles": [ + { + "depth_for_recall": [ + [ + 0.9, + 24 + ], + [ + 0.95, + 36 + ], + [ + 0.99, + 80 + ], + [ + 1.0, + 713 + ] + ], + "first": "binary", + "first_bytes": 48, + "k": 10, + "median_full_depth": 15.0, + "metric": "cosine", + "neighbour_band": 0.024774938821792603, + "queries": 500, + "rank_noise": 0.018597556555114886, + "recall_at_k": 0.7190000000000004, + "vectors": 20648 + }, + { + "depth_for_recall": [ + [ + 0.9, + 22 + ], + [ + 0.95, + 34 + ], + [ + 0.99, + 79 + ], + [ + 1.0, + 395 + ] + ], + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "median_full_depth": 15.0, + "metric": "cosine", + "neighbour_band": 0.024774938821792603, + "queries": 500, + "rank_noise": 0.01770042272474518, + "recall_at_k": 0.7246000000000006, + "vectors": 20648 + }, + { + "depth_for_recall": [ + [ + 0.9, + 21 + ], + [ + 0.95, + 28 + ], + [ + 0.99, + 51 + ], + [ + 1.0, + 123 + ] + ], + "first": "pq48", + "first_bytes": 48, + "k": 10, + "median_full_depth": 14.0, + "metric": "cosine", + "neighbour_band": 0.024774938821792603, + "queries": 500, + "rank_noise": 0.016073076858259247, + "recall_at_k": 0.775, + "vectors": 20648 + }, + { + "depth_for_recall": [ + [ + 0.9, + 22 + ], + [ + 0.95, + 32 + ], + [ + 0.99, + 73 + ], + [ + 1.0, + 483 + ] + ], + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "median_full_depth": 15.0, + "metric": "cosine", + "neighbour_band": 0.024774938821792603, + "queries": 500, + "rank_noise": 0.017551227488585123, + "recall_at_k": 0.7222000000000008, + "vectors": 20648 + }, + { + "depth_for_recall": [ + [ + 0.9, + 16 + ], + [ + 0.95, + 22 + ], + [ + 0.99, + 41 + ], + [ + 1.0, + 109 + ] + ], + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "median_full_depth": 11.0, + "metric": "cosine", + "neighbour_band": 0.024774938821792603, + "queries": 500, + "rank_noise": 0.013723295336378635, + "recall_at_k": 0.8166, + "vectors": 20648 + }, + { + "depth_for_recall": [ + [ + 0.9, + 15 + ], + [ + 0.95, + 20 + ], + [ + 0.99, + 39 + ], + [ + 1.0, + 116 + ] + ], + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "median_full_depth": 11.0, + "metric": "cosine", + "neighbour_band": 0.024774938821792603, + "queries": 500, + "rank_noise": 0.013181060878126443, + "recall_at_k": 0.8169999999999996, + "vectors": 20648 + }, + { + "depth_for_recall": [ + [ + 0.9, + 34 + ], + [ + 0.95, + 58 + ], + [ + 0.99, + 146 + ], + [ + 1.0, + 361 + ] + ], + "first": "pq24", + "first_bytes": 24, + "k": 10, + "median_full_depth": 21.0, + "metric": "cosine", + "neighbour_band": 0.024774938821792603, + "queries": 500, + "rank_noise": 0.02516490676753467, + "recall_at_k": 0.6399999999999997, + "vectors": 20648 + }, + { + "depth_for_recall": [ + [ + 0.9, + 28 + ], + [ + 0.95, + 41 + ], + [ + 0.99, + 85 + ], + [ + 1.0, + 284 + ] + ], + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "median_full_depth": 18.0, + "metric": "cosine", + "neighbour_band": 0.024774938821792603, + "queries": 500, + "rank_noise": 0.022116097828018245, + "recall_at_k": 0.6920000000000004, + "vectors": 20648 + }, + { + "depth_for_recall": [ + [ + 0.9, + 27 + ], + [ + 0.95, + 43 + ], + [ + 0.99, + 94 + ], + [ + 1.0, + 278 + ] + ], + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "median_full_depth": 18.0, + "metric": "cosine", + "neighbour_band": 0.024774938821792603, + "queries": 500, + "rank_noise": 0.022573340182736212, + "recall_at_k": 0.6912, + "vectors": 20648 + }, + { + "depth_for_recall": [ + [ + 0.9, + 11 + ], + [ + 0.95, + 16 + ], + [ + 0.99, + 26 + ], + [ + 1.0, + 53 + ] + ], + "first": "pq96", + "first_bytes": 96, + "k": 10, + "median_full_depth": 10.0, + "metric": "cosine", + "neighbour_band": 0.024774938821792603, + "queries": 500, + "rank_noise": 0.007792641547558008, + "recall_at_k": 0.8969999999999995, + "vectors": 20648 + }, + { + "depth_for_recall": [ + [ + 0.9, + 10 + ], + [ + 0.95, + 12 + ], + [ + 0.99, + 21 + ], + [ + 1.0, + 44 + ] + ], + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "median_full_depth": 10.0, + "metric": "cosine", + "neighbour_band": 0.024774938821792603, + "queries": 500, + "rank_noise": 0.005552380597888168, + "recall_at_k": 0.9297999999999996, + "vectors": 20648 + } + ] +} diff --git a/experiments/2026-11-hybrid-adaptive-matrix/results/fiqa-bge-small-k10.json b/experiments/2026-11-hybrid-adaptive-matrix/results/fiqa-bge-small-k10.json new file mode 100644 index 0000000..0c09cec --- /dev/null +++ b/experiments/2026-11-hybrid-adaptive-matrix/results/fiqa-bge-small-k10.json @@ -0,0 +1,3900 @@ +{ + "cells": [ + { + "bytes_read_per_query": 3421152.0, + "depth": 0, + "doc_recall_at_k": 0.3947765026468731, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.3334523978137343, + "p10_recall_at_k": 0.3, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 648, + "recall_at_k": 0.5367283950617282, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.49537037037037035, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3428992.0, + "depth": 20, + "doc_recall_at_k": 0.41408499764981255, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.3754630596976833, + "p10_recall_at_k": 0.5, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.7172839506172839, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9475308641975309, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3440752.0, + "depth": 50, + "doc_recall_at_k": 0.43150970257451754, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.38323410734130464, + "p10_recall_at_k": 0.7, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.873765432098762, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9675925925925926, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3460352.0, + "depth": 100, + "doc_recall_at_k": 0.43791586071678656, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.3869491884116496, + "p10_recall_at_k": 0.8, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.9370370370370331, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9799382716049383, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3433343.3209876544, + "depth": 31, + "doc_recall_at_k": 0.41945980545517586, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 31.10030864197531, + "metric": "cosine", + "ndcg_at_k": 0.377737367379139, + "p10_recall_at_k": 0.6, + "p90_depth": 45, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 648, + "recall_at_k": 0.8050925925925909, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9645061728395061, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3457353.3209876544, + "depth": 92, + "doc_recall_at_k": 0.4390904149006001, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 92.3503086419753, + "metric": "cosine", + "ndcg_at_k": 0.3870814156027092, + "p10_recall_at_k": 0.8, + "p90_depth": 152, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 648, + "recall_at_k": 0.9353395061728348, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9799382716049383, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3519395.185185185, + "depth": 251, + "doc_recall_at_k": 0.4379526038090853, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 250.62037037037038, + "metric": "cosine", + "ndcg_at_k": 0.38674528316979384, + "p10_recall_at_k": 0.9, + "p90_depth": 453, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 648, + "recall_at_k": 0.97083333333333, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9814814814814815, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3451872.0, + "depth": 20, + "doc_recall_at_k": 0.41330726886282454, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.37522166756712916, + "p10_recall_at_k": 0.5, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.7179012345679012, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9645061728395061, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3497952.0, + "depth": 50, + "doc_recall_at_k": 0.43210983974872874, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.3834727426221282, + "p10_recall_at_k": 0.7, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.8793209876543183, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9861111111111112, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3574752.0, + "depth": 100, + "doc_recall_at_k": 0.43637265084024335, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.3866850426308114, + "p10_recall_at_k": 0.8, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.9478395061728367, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9984567901234568, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3468922.074074074, + "depth": 31, + "doc_recall_at_k": 0.4183023980477685, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 31.10030864197531, + "metric": "cosine", + "ndcg_at_k": 0.37742668658172396, + "p10_recall_at_k": 0.6, + "p90_depth": 45, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 648, + "recall_at_k": 0.8061728395061714, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9814814814814815, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3563002.074074074, + "depth": 92, + "doc_recall_at_k": 0.4366408754140237, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 92.3503086419753, + "metric": "cosine", + "ndcg_at_k": 0.3863786242657091, + "p10_recall_at_k": 0.8, + "p90_depth": 152, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 648, + "recall_at_k": 0.9475308641975275, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9984567901234568, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3806104.888888889, + "depth": 251, + "doc_recall_at_k": 0.43666659557863263, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 250.62037037037038, + "metric": "cosine", + "ndcg_at_k": 0.3864520107139816, + "p10_recall_at_k": 1.0, + "p90_depth": 453, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 648, + "recall_at_k": 0.9899691358024683, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3421152.0, + "depth": 0, + "doc_recall_at_k": 0.40706942234720006, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.344364230170545, + "p10_recall_at_k": 0.4, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 648, + "recall_at_k": 0.6001543209876543, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.5432098765432098, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3428992.0, + "depth": 20, + "doc_recall_at_k": 0.42054929381318257, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.37795915453103246, + "p10_recall_at_k": 0.6, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.7831790123456787, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9629629629629629, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3440752.0, + "depth": 50, + "doc_recall_at_k": 0.4348380143519032, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.38515664675390215, + "p10_recall_at_k": 0.8, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.9270061728395023, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9799382716049383, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3460352.0, + "depth": 100, + "doc_recall_at_k": 0.4379526038090853, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.38676789222925323, + "p10_recall_at_k": 0.9, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.9628086419753055, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9814814814814815, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3432103.1975308643, + "depth": 28, + "doc_recall_at_k": 0.42434301809301794, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 27.936728395061728, + "metric": "cosine", + "ndcg_at_k": 0.38171005471678926, + "p10_recall_at_k": 0.7, + "p90_depth": 41, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 648, + "recall_at_k": 0.8557098765432073, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9753086419753086, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3450986.345679012, + "depth": 76, + "doc_recall_at_k": 0.4367308959901552, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 76.10802469135803, + "metric": "cosine", + "ndcg_at_k": 0.3863485226498704, + "p10_recall_at_k": 0.9, + "p90_depth": 132, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 648, + "recall_at_k": 0.9563271604938228, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9814814814814815, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3496848.5308641978, + "depth": 193, + "doc_recall_at_k": 0.4394958136856284, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 193.1033950617284, + "metric": "cosine", + "ndcg_at_k": 0.3872220880286448, + "p10_recall_at_k": 0.9, + "p90_depth": 351, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 648, + "recall_at_k": 0.9756172839506142, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9814814814814815, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3451872.0, + "depth": 20, + "doc_recall_at_k": 0.42003489052100157, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.3777789702746299, + "p10_recall_at_k": 0.6, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.7844135802469133, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9814814814814815, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3497952.0, + "depth": 50, + "doc_recall_at_k": 0.4323027409832965, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.3845539233322665, + "p10_recall_at_k": 0.8, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.9368827160493798, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9984567901234568, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3574752.0, + "depth": 100, + "doc_recall_at_k": 0.4367547790001494, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.38650859474539234, + "p10_recall_at_k": 0.9, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.9791666666666651, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3464062.814814815, + "depth": 28, + "doc_recall_at_k": 0.4212565983399315, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 27.936728395061728, + "metric": "cosine", + "ndcg_at_k": 0.3808583172808903, + "p10_recall_at_k": 0.7, + "p90_depth": 41, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 648, + "recall_at_k": 0.8584876543209853, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9938271604938271, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3538053.925925926, + "depth": 76, + "doc_recall_at_k": 0.4355330711812194, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 76.10802469135803, + "metric": "cosine", + "ndcg_at_k": 0.3861299659846328, + "p10_recall_at_k": 0.9, + "p90_depth": 132, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 648, + "recall_at_k": 0.972376543209874, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3717758.814814815, + "depth": 193, + "doc_recall_at_k": 0.4379893469013839, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 193.1033950617284, + "metric": "cosine", + "ndcg_at_k": 0.3868114953505997, + "p10_recall_at_k": 1.0, + "p90_depth": 351, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 648, + "recall_at_k": 0.99645061728395, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3421152.0, + "depth": 0, + "doc_recall_at_k": 0.4089782259921148, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.3532630384905834, + "p10_recall_at_k": 0.4, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 648, + "recall_at_k": 0.6418209876543218, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.5925925925925926, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3428992.0, + "depth": 20, + "doc_recall_at_k": 0.43074238433034717, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.38113991606792236, + "p10_recall_at_k": 0.7, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.8376543209876525, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9722222222222222, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3440752.0, + "depth": 50, + "doc_recall_at_k": 0.4387242087473568, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.3869095924635434, + "p10_recall_at_k": 0.9, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.949845679012342, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9814814814814815, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3460352.0, + "depth": 100, + "doc_recall_at_k": 0.4394958136856284, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.3872307841545922, + "p10_recall_at_k": 0.9, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.9736111111111083, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9814814814814815, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3430512.814814815, + "depth": 24, + "doc_recall_at_k": 0.43481903042088227, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 23.87962962962963, + "metric": "cosine", + "ndcg_at_k": 0.38477022184576526, + "p10_recall_at_k": 0.7, + "p90_depth": 34, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 648, + "recall_at_k": 0.8709876543209836, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9722222222222222, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3442754.345679012, + "depth": 55, + "doc_recall_at_k": 0.43723243920003174, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 55.108024691358025, + "metric": "cosine", + "ndcg_at_k": 0.38648478725036167, + "p10_recall_at_k": 0.9, + "p90_depth": 89, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 648, + "recall_at_k": 0.9572530864197489, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9814814814814815, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3469322.024691358, + "depth": 123, + "doc_recall_at_k": 0.4394958136856284, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 122.88271604938272, + "metric": "cosine", + "ndcg_at_k": 0.3872220880286448, + "p10_recall_at_k": 0.9, + "p90_depth": 220, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 648, + "recall_at_k": 0.9765432098765405, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9814814814814815, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3451872.0, + "depth": 20, + "doc_recall_at_k": 0.4302279810381662, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.3810573456364275, + "p10_recall_at_k": 0.7, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.840277777777776, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9907407407407407, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3497952.0, + "depth": 50, + "doc_recall_at_k": 0.4366518983417132, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.38628352663861104, + "p10_recall_at_k": 0.9, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.9646604938271585, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3574752.0, + "depth": 100, + "doc_recall_at_k": 0.438503750193565, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.3870295309417207, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.9929012345679007, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3457831.111111111, + "depth": 24, + "doc_recall_at_k": 0.43176935376009445, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 23.87962962962963, + "metric": "cosine", + "ndcg_at_k": 0.3839144126277606, + "p10_recall_at_k": 0.7, + "p90_depth": 34, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 648, + "recall_at_k": 0.8749999999999967, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9907407407407407, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3505797.925925926, + "depth": 55, + "doc_recall_at_k": 0.4362403757079683, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 55.108024691358025, + "metric": "cosine", + "ndcg_at_k": 0.386305810088604, + "p10_recall_at_k": 0.9, + "p90_depth": 89, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 648, + "recall_at_k": 0.9737654320987633, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3609899.8518518517, + "depth": 123, + "doc_recall_at_k": 0.438503750193565, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 122.88271604938272, + "metric": "cosine", + "ndcg_at_k": 0.38702083481577326, + "p10_recall_at_k": 1.0, + "p90_depth": 220, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 648, + "recall_at_k": 0.9973765432098761, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3421152.0, + "depth": 0, + "doc_recall_at_k": 0.39839380441232297, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.33833967178248164, + "p10_recall_at_k": 0.4, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 648, + "recall_at_k": 0.5891975308641977, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.5432098765432098, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3428992.0, + "depth": 20, + "doc_recall_at_k": 0.4172920572226127, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.3751006030842581, + "p10_recall_at_k": 0.6, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.7782407407407403, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9598765432098766, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3440752.0, + "depth": 50, + "doc_recall_at_k": 0.4323027409832965, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.3842798414727766, + "p10_recall_at_k": 0.8, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.9220679012345644, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9768518518518519, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3460352.0, + "depth": 100, + "doc_recall_at_k": 0.43784972315064896, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.38664690724626144, + "p10_recall_at_k": 0.9, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.9615740740740708, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9799382716049383, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3431994.9135802467, + "depth": 28, + "doc_recall_at_k": 0.42413245191022964, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 27.660493827160494, + "metric": "cosine", + "ndcg_at_k": 0.38011809765988397, + "p10_recall_at_k": 0.7, + "p90_depth": 40, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 648, + "recall_at_k": 0.844753086419751, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9691358024691358, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3449919.839506173, + "depth": 73, + "doc_recall_at_k": 0.4360450249339138, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 73.38734567901234, + "metric": "cosine", + "ndcg_at_k": 0.3856656335272849, + "p10_recall_at_k": 0.9, + "p90_depth": 121, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 648, + "recall_at_k": 0.9550925925925882, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9799382716049383, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3492560.7283950616, + "depth": 182, + "doc_recall_at_k": 0.4394958136856284, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 182.16512345679013, + "metric": "cosine", + "ndcg_at_k": 0.3872220880286448, + "p10_recall_at_k": 0.9, + "p90_depth": 339, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 648, + "recall_at_k": 0.9746913580246884, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9814814814814815, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3451872.0, + "depth": 20, + "doc_recall_at_k": 0.4188352670991559, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.3758054164994256, + "p10_recall_at_k": 0.6, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.7797839506172837, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9783950617283951, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3497952.0, + "depth": 50, + "doc_recall_at_k": 0.4297674676146898, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.38362268228034296, + "p10_recall_at_k": 0.8, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.931481481481479, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9953703703703703, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3574752.0, + "depth": 100, + "doc_recall_at_k": 0.43742350327998475, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.3866795919477866, + "p10_recall_at_k": 0.9, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.9770061728395046, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9984567901234568, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3463638.5185185187, + "depth": 28, + "doc_recall_at_k": 0.4222034395645506, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 27.660493827160494, + "metric": "cosine", + "ndcg_at_k": 0.3795720133213241, + "p10_recall_at_k": 0.7, + "p90_depth": 40, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 648, + "recall_at_k": 0.8481481481481462, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9876543209876543, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3533874.962962963, + "depth": 73, + "doc_recall_at_k": 0.4345385581496693, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 73.38734567901234, + "metric": "cosine", + "ndcg_at_k": 0.38527976813108805, + "p10_recall_at_k": 0.9, + "p90_depth": 121, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 648, + "recall_at_k": 0.9694444444444416, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9984567901234568, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3700957.6296296297, + "depth": 182, + "doc_recall_at_k": 0.438503750193565, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 182.16512345679013, + "metric": "cosine", + "ndcg_at_k": 0.38703929958328265, + "p10_recall_at_k": 1.0, + "p90_depth": 339, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 648, + "recall_at_k": 0.9955246913580242, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3421152.0, + "depth": 0, + "doc_recall_at_k": 0.41163358246691567, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.35570261268089776, + "p10_recall_at_k": 0.5, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 648, + "recall_at_k": 0.6628086419753089, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.6219135802469136, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3428992.0, + "depth": 20, + "doc_recall_at_k": 0.4358620160703493, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.38555498855764525, + "p10_recall_at_k": 0.7, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.852314814814813, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9783950617283951, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3440752.0, + "depth": 50, + "doc_recall_at_k": 0.4380910970031339, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.38673333451802006, + "p10_recall_at_k": 0.9, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.9594135802469098, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9814814814814815, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3460352.0, + "depth": 100, + "doc_recall_at_k": 0.4394958136856284, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.3872220880286448, + "p10_recall_at_k": 0.9, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.9759259259259228, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9814814814814815, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3429812.296296296, + "depth": 22, + "doc_recall_at_k": 0.4345245675106786, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 22.09259259259259, + "metric": "cosine", + "ndcg_at_k": 0.3847414769188782, + "p10_recall_at_k": 0.7, + "p90_depth": 31, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 648, + "recall_at_k": 0.8717592592592562, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9783950617283951, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3440185.777777778, + "depth": 49, + "doc_recall_at_k": 0.43944140564510925, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 48.55555555555556, + "metric": "cosine", + "ndcg_at_k": 0.38713617637081815, + "p10_recall_at_k": 0.9, + "p90_depth": 76, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 648, + "recall_at_k": 0.9586419753086377, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9814814814814815, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3461069.4567901236, + "depth": 102, + "doc_recall_at_k": 0.4394958136856284, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 101.83024691358025, + "metric": "cosine", + "ndcg_at_k": 0.38719002897220167, + "p10_recall_at_k": 0.9, + "p90_depth": 177, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 648, + "recall_at_k": 0.9759259259259228, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9814814814814815, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3451872.0, + "depth": 20, + "doc_recall_at_k": 0.4363396762702317, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.3858449162881483, + "p10_recall_at_k": 0.7, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.8546296296296282, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9969135802469136, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3497952.0, + "depth": 50, + "doc_recall_at_k": 0.43792207877856026, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.3868371809719727, + "p10_recall_at_k": 0.9, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.9737654320987633, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3574752.0, + "depth": 100, + "doc_recall_at_k": 0.438503750193565, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.38702083481577326, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.9961419753086411, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3455086.222222222, + "depth": 22, + "doc_recall_at_k": 0.43448782441837996, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 22.09259259259259, + "metric": "cosine", + "ndcg_at_k": 0.38500211805503864, + "p10_recall_at_k": 0.7, + "p90_depth": 31, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 648, + "recall_at_k": 0.8746913580246888, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9969135802469136, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3495733.3333333335, + "depth": 49, + "doc_recall_at_k": 0.43927238742053554, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 48.55555555555556, + "metric": "cosine", + "ndcg_at_k": 0.38731568704176716, + "p10_recall_at_k": 0.9, + "p90_depth": 76, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 648, + "recall_at_k": 0.9736111111111087, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3577563.2592592593, + "depth": 102, + "doc_recall_at_k": 0.438503750193565, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 101.83024691358025, + "metric": "cosine", + "ndcg_at_k": 0.38698877575933016, + "p10_recall_at_k": 1.0, + "p90_depth": 177, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 648, + "recall_at_k": 0.9967592592592587, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3421152.0, + "depth": 0, + "doc_recall_at_k": 0.41665150264687306, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.35119217403719766, + "p10_recall_at_k": 0.5, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 648, + "recall_at_k": 0.6699074074074076, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.6049382716049383, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3428992.0, + "depth": 20, + "doc_recall_at_k": 0.42998976332309674, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.38207983777003307, + "p10_recall_at_k": 0.7, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.8631172839506143, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9753086419753086, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3440752.0, + "depth": 50, + "doc_recall_at_k": 0.436366526991527, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.38578758396347024, + "p10_recall_at_k": 0.9, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.9603395061728363, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9814814814814815, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3460352.0, + "depth": 100, + "doc_recall_at_k": 0.4394958136856284, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.3872220880286448, + "p10_recall_at_k": 0.9, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.9743827160493796, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9814814814814815, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3429707.641975309, + "depth": 22, + "doc_recall_at_k": 0.4250117423728535, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 21.825617283950617, + "metric": "cosine", + "ndcg_at_k": 0.3797020407670158, + "p10_recall_at_k": 0.7, + "p90_depth": 30, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 648, + "recall_at_k": 0.8780864197530825, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9783950617283951, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3439383.024691358, + "depth": 47, + "doc_recall_at_k": 0.4358006833701278, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 46.507716049382715, + "metric": "cosine", + "ndcg_at_k": 0.38545379037453126, + "p10_recall_at_k": 0.9, + "p90_depth": 74, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 648, + "recall_at_k": 0.958641975308638, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9814814814814815, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3458601.3086419753, + "depth": 96, + "doc_recall_at_k": 0.4394958136856284, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 95.53395061728395, + "metric": "cosine", + "ndcg_at_k": 0.3872220880286448, + "p10_recall_at_k": 0.9, + "p90_depth": 164, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 648, + "recall_at_k": 0.9751543209876514, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9814814814814815, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3451872.0, + "depth": 20, + "doc_recall_at_k": 0.4302837080614859, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.3823428854209756, + "p10_recall_at_k": 0.7, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.8663580246913554, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9938271604938271, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3497952.0, + "depth": 50, + "doc_recall_at_k": 0.4366604717299161, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.3860046470906581, + "p10_recall_at_k": 0.9, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.9766975308641961, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3574752.0, + "depth": 100, + "doc_recall_at_k": 0.438503750193565, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.38702083481577326, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.9942901234567896, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3454676.1481481483, + "depth": 22, + "doc_recall_at_k": 0.4255628887573332, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 21.825617283950617, + "metric": "cosine", + "ndcg_at_k": 0.38011364149266347, + "p10_recall_at_k": 0.7, + "p90_depth": 30, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 648, + "recall_at_k": 0.8828703703703671, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9969135802469136, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3492587.8518518517, + "depth": 47, + "doc_recall_at_k": 0.4364032700838256, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 46.507716049382715, + "metric": "cosine", + "ndcg_at_k": 0.3858435298461383, + "p10_recall_at_k": 0.9, + "p90_depth": 74, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 648, + "recall_at_k": 0.9748456790123434, + "rescore_ms": 3, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3567892.1481481483, + "depth": 96, + "doc_recall_at_k": 0.438503750193565, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 95.53395061728395, + "metric": "cosine", + "ndcg_at_k": 0.38702083481577326, + "p10_recall_at_k": 1.0, + "p90_depth": 164, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 648, + "recall_at_k": 0.9955246913580243, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 71274 + }, + { + "bytes_read_per_query": 1710576.0, + "depth": 0, + "doc_recall_at_k": 0.3450280986160616, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 648, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.2745208420960961, + "p10_recall_at_k": 0.2, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 648, + "recall_at_k": 0.4453703703703707, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.37808641975308643, + "vectors": 71274 + }, + { + "bytes_read_per_query": 1718416.0, + "depth": 20, + "doc_recall_at_k": 0.3940285922924811, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 648, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.3592648687065969, + "p10_recall_at_k": 0.4, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.6109567901234567, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.8672839506172839, + "vectors": 71274 + }, + { + "bytes_read_per_query": 1730176.0, + "depth": 50, + "doc_recall_at_k": 0.40614472119101735, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 648, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.36742712091667346, + "p10_recall_at_k": 0.6, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.800308641975307, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9382716049382716, + "vectors": 71274 + }, + { + "bytes_read_per_query": 1749776.0, + "depth": 100, + "doc_recall_at_k": 0.4213006343793381, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 648, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.376445895489762, + "p10_recall_at_k": 0.7, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.8953703703703669, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9675925925925926, + "vectors": 71274 + }, + { + "bytes_read_per_query": 1729443.4197530865, + "depth": 48, + "doc_recall_at_k": 0.4045904883867847, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 648, + "mean_depth": 48.13117283950617, + "metric": "cosine", + "ndcg_at_k": 0.36850568247848775, + "p10_recall_at_k": 0.6, + "p90_depth": 75, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 648, + "recall_at_k": 0.787654320987654, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9475308641975309, + "vectors": 71274 + }, + { + "bytes_read_per_query": 1783723.3209876544, + "depth": 187, + "doc_recall_at_k": 0.4302451278145723, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 648, + "mean_depth": 186.60030864197532, + "metric": "cosine", + "ndcg_at_k": 0.383199946206717, + "p10_recall_at_k": 0.9, + "p90_depth": 341, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 648, + "recall_at_k": 0.9456790123456748, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9768518518518519, + "vectors": 71274 + }, + { + "bytes_read_per_query": 1913961.086419753, + "depth": 519, + "doc_recall_at_k": 0.43911001121649257, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 648, + "mean_depth": 518.8395061728395, + "metric": "cosine", + "ndcg_at_k": 0.3869111162459173, + "p10_recall_at_k": 0.9, + "p90_depth": 1000, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 648, + "recall_at_k": 0.9748456790123428, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9799382716049383, + "vectors": 71274 + }, + { + "bytes_read_per_query": 1741296.0, + "depth": 20, + "doc_recall_at_k": 0.3924853824159379, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 648, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.3591083077698998, + "p10_recall_at_k": 0.4, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.6109567901234567, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.8780864197530864, + "vectors": 71274 + }, + { + "bytes_read_per_query": 1787376.0, + "depth": 50, + "doc_recall_at_k": 0.40730212859842474, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 648, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.3683648980834326, + "p10_recall_at_k": 0.6, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.8023148148148134, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9552469135802469, + "vectors": 71274 + }, + { + "bytes_read_per_query": 1864176.0, + "depth": 100, + "doc_recall_at_k": 0.4211291666152777, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 648, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.37669196801223515, + "p10_recall_at_k": 0.7, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.9023148148148125, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9845679012345679, + "vectors": 71274 + }, + { + "bytes_read_per_query": 1784505.4814814816, + "depth": 48, + "doc_recall_at_k": 0.4040760850946037, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 648, + "mean_depth": 48.13117283950617, + "metric": "cosine", + "ndcg_at_k": 0.3688475783638241, + "p10_recall_at_k": 0.6, + "p90_depth": 75, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 648, + "recall_at_k": 0.7893518518518516, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9614197530864198, + "vectors": 71274 + }, + { + "bytes_read_per_query": 1997194.0740740742, + "depth": 187, + "doc_recall_at_k": 0.42976746761468987, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 648, + "mean_depth": 186.60030864197532, + "metric": "cosine", + "ndcg_at_k": 0.38333183004409294, + "p10_recall_at_k": 0.9, + "p90_depth": 341, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 648, + "recall_at_k": 0.9591049382716019, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9953703703703703, + "vectors": 71274 + }, + { + "bytes_read_per_query": 2507513.4814814813, + "depth": 519, + "doc_recall_at_k": 0.43888955266270085, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 648, + "mean_depth": 518.8395061728395, + "metric": "cosine", + "ndcg_at_k": 0.38697205883055885, + "p10_recall_at_k": 1.0, + "p90_depth": 1000, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 648, + "recall_at_k": 0.9944444444444439, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9984567901234568, + "vectors": 71274 + }, + { + "bytes_read_per_query": 1710576.0, + "depth": 0, + "doc_recall_at_k": 0.3739541194402305, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 648, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.3087153735694904, + "p10_recall_at_k": 0.3, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 648, + "recall_at_k": 0.5061728395061725, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.4089506172839506, + "vectors": 71274 + }, + { + "bytes_read_per_query": 1718416.0, + "depth": 20, + "doc_recall_at_k": 0.398663977483422, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 648, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.3622255040308618, + "p10_recall_at_k": 0.5, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.6893518518518519, + "rescore_ms": 44, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9104938271604939, + "vectors": 71274 + }, + { + "bytes_read_per_query": 1730176.0, + "depth": 50, + "doc_recall_at_k": 0.4181022424077979, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 648, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.3725075778999628, + "p10_recall_at_k": 0.7, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.8637345679012315, + "rescore_ms": 110, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9645061728395061, + "vectors": 71274 + }, + { + "bytes_read_per_query": 1749776.0, + "depth": 100, + "doc_recall_at_k": 0.43162544331525815, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 648, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.38218346444992873, + "p10_recall_at_k": 0.8, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.9322530864197491, + "rescore_ms": 97, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9737654320987654, + "vectors": 71274 + }, + { + "bytes_read_per_query": 1724150.2098765431, + "depth": 35, + "doc_recall_at_k": 0.40889264199449377, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 648, + "mean_depth": 34.62808641975309, + "metric": "cosine", + "ndcg_at_k": 0.369533673922182, + "p10_recall_at_k": 0.6, + "p90_depth": 51, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 648, + "recall_at_k": 0.8040123456790097, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9459876543209876, + "vectors": 71274 + }, + { + "bytes_read_per_query": 1752918.6543209876, + "depth": 108, + "doc_recall_at_k": 0.43258994948809765, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 648, + "mean_depth": 108.01697530864197, + "metric": "cosine", + "ndcg_at_k": 0.3834138671705348, + "p10_recall_at_k": 0.8, + "p90_depth": 181, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 648, + "recall_at_k": 0.9373456790123414, + "rescore_ms": 34, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9783950617283951, + "vectors": 71274 + }, + { + "bytes_read_per_query": 1825304.3580246912, + "depth": 293, + "doc_recall_at_k": 0.4384155667720482, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 648, + "mean_depth": 292.67438271604937, + "metric": "cosine", + "ndcg_at_k": 0.3867371532966745, + "p10_recall_at_k": 0.9, + "p90_depth": 527, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 648, + "recall_at_k": 0.9716049382716014, + "rescore_ms": 273, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9799382716049383, + "vectors": 71274 + }, + { + "bytes_read_per_query": 1741296.0, + "depth": 20, + "doc_recall_at_k": 0.39943558242169364, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 648, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.36287278683169527, + "p10_recall_at_k": 0.5, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.6898148148148148, + "rescore_ms": 63, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9259259259259259, + "vectors": 71274 + }, + { + "bytes_read_per_query": 1787376.0, + "depth": 50, + "doc_recall_at_k": 0.4188371042537709, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 648, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.3730531869365311, + "p10_recall_at_k": 0.7, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.8694444444444421, + "rescore_ms": 101, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9830246913580247, + "vectors": 71274 + }, + { + "bytes_read_per_query": 1864176.0, + "depth": 100, + "doc_recall_at_k": 0.42986177488492305, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 648, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.3816884934767029, + "p10_recall_at_k": 0.8, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.9442901234567876, + "rescore_ms": 174, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9922839506172839, + "vectors": 71274 + }, + { + "bytes_read_per_query": 1763764.7407407407, + "depth": 35, + "doc_recall_at_k": 0.40812103705622216, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 648, + "mean_depth": 34.62808641975309, + "metric": "cosine", + "ndcg_at_k": 0.36966378818550005, + "p10_recall_at_k": 0.6, + "p90_depth": 51, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 648, + "recall_at_k": 0.8063271604938249, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9629629629629629, + "vectors": 71274 + }, + { + "bytes_read_per_query": 1876490.0740740742, + "depth": 108, + "doc_recall_at_k": 0.43103204237463505, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 648, + "mean_depth": 108.01697530864197, + "metric": "cosine", + "ndcg_at_k": 0.38289460511853085, + "p10_recall_at_k": 0.8, + "p90_depth": 181, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 648, + "recall_at_k": 0.951234567901232, + "rescore_ms": 107, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9969135802469136, + "vectors": 71274 + }, + { + "bytes_read_per_query": 2160123.8518518517, + "depth": 293, + "doc_recall_at_k": 0.43742350327998475, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 648, + "mean_depth": 292.67438271604937, + "metric": "cosine", + "ndcg_at_k": 0.38653590008380295, + "p10_recall_at_k": 1.0, + "p90_depth": 527, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 648, + "recall_at_k": 0.9921296296296287, + "rescore_ms": 689, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9984567901234568, + "vectors": 71274 + }, + { + "bytes_read_per_query": 1710576.0, + "depth": 0, + "doc_recall_at_k": 0.3654541232087529, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 648, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.2980364376778953, + "p10_recall_at_k": 0.3, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 648, + "recall_at_k": 0.4907407407407402, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.41975308641975306, + "vectors": 71274 + }, + { + "bytes_read_per_query": 1718416.0, + "depth": 20, + "doc_recall_at_k": 0.4036522590920738, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 648, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.36464480724741627, + "p10_recall_at_k": 0.4, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.6859567901234562, + "rescore_ms": 123, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9135802469135802, + "vectors": 71274 + }, + { + "bytes_read_per_query": 1730176.0, + "depth": 50, + "doc_recall_at_k": 0.415928888498333, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 648, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.3732646690835015, + "p10_recall_at_k": 0.7, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.8592592592592562, + "rescore_ms": 157, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9645061728395061, + "vectors": 71274 + }, + { + "bytes_read_per_query": 1749776.0, + "depth": 100, + "doc_recall_at_k": 0.4311295057822836, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 648, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.3810314085718142, + "p10_recall_at_k": 0.8, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.9297839506172803, + "rescore_ms": 207, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9783950617283951, + "vectors": 71274 + }, + { + "bytes_read_per_query": 1724279.061728395, + "depth": 35, + "doc_recall_at_k": 0.41216517107720807, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 648, + "mean_depth": 34.95679012345679, + "metric": "cosine", + "ndcg_at_k": 0.37161895597671235, + "p10_recall_at_k": 0.6, + "p90_depth": 52, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 648, + "recall_at_k": 0.8035493827160483, + "rescore_ms": 1, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9567901234567902, + "vectors": 71274 + }, + { + "bytes_read_per_query": 1752619.8148148148, + "depth": 107, + "doc_recall_at_k": 0.42763453110675326, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 648, + "mean_depth": 107.25462962962963, + "metric": "cosine", + "ndcg_at_k": 0.3799678792836182, + "p10_recall_at_k": 0.8, + "p90_depth": 178, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 648, + "recall_at_k": 0.9348765432098725, + "rescore_ms": 88, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9783950617283951, + "vectors": 71274 + }, + { + "bytes_read_per_query": 1822142.950617284, + "depth": 285, + "doc_recall_at_k": 0.4371809988708136, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 648, + "mean_depth": 284.6095679012346, + "metric": "cosine", + "ndcg_at_k": 0.38563372543597674, + "p10_recall_at_k": 0.9, + "p90_depth": 504, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 648, + "recall_at_k": 0.9708333333333297, + "rescore_ms": 549, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9814814814814815, + "vectors": 71274 + }, + { + "bytes_read_per_query": 1741296.0, + "depth": 20, + "doc_recall_at_k": 0.4029663880358324, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 648, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.36448739607011454, + "p10_recall_at_k": 0.4, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.6864197530864192, + "rescore_ms": 170, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9305555555555556, + "vectors": 71274 + }, + { + "bytes_read_per_query": 1787376.0, + "depth": 50, + "doc_recall_at_k": 0.41541448520615193, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 648, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.373343307929357, + "p10_recall_at_k": 0.7, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.8645061728395038, + "rescore_ms": 226, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9830246913580247, + "vectors": 71274 + }, + { + "bytes_read_per_query": 1864176.0, + "depth": 100, + "doc_recall_at_k": 0.42826354458298904, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 648, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.38037611898985396, + "p10_recall_at_k": 0.8, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.9418209876543188, + "rescore_ms": 335, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9969135802469136, + "vectors": 71274 + }, + { + "bytes_read_per_query": 1764269.6296296297, + "depth": 35, + "doc_recall_at_k": 0.4122509049592382, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 648, + "mean_depth": 34.95679012345679, + "metric": "cosine", + "ndcg_at_k": 0.3718448402253727, + "p10_recall_at_k": 0.6, + "p90_depth": 52, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 648, + "recall_at_k": 0.8052469135802459, + "rescore_ms": 4, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9753086419753086, + "vectors": 71274 + }, + { + "bytes_read_per_query": 1875319.111111111, + "depth": 107, + "doc_recall_at_k": 0.4260545781379114, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 648, + "mean_depth": 107.25462962962963, + "metric": "cosine", + "ndcg_at_k": 0.37973861088658234, + "p10_recall_at_k": 0.8, + "p90_depth": 178, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 648, + "recall_at_k": 0.9495370370370347, + "rescore_ms": 209, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9969135802469136, + "vectors": 71274 + }, + { + "bytes_read_per_query": 2147736.296296296, + "depth": 285, + "doc_recall_at_k": 0.43618893537875025, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 648, + "mean_depth": 284.6095679012346, + "metric": "cosine", + "ndcg_at_k": 0.3854864299238348, + "p10_recall_at_k": 1.0, + "p90_depth": 504, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 648, + "recall_at_k": 0.9912037037037026, + "rescore_ms": 1367, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 71274 + }, + { + "bytes_read_per_query": 6842304.0, + "depth": 0, + "doc_recall_at_k": 0.4299113780595262, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 648, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.37626699523918095, + "p10_recall_at_k": 0.7, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 648, + "recall_at_k": 0.8165123456790103, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.7916666666666666, + "vectors": 71274 + }, + { + "bytes_read_per_query": 6850144.0, + "depth": 20, + "doc_recall_at_k": 0.43989018954296727, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 648, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.38720727594391846, + "p10_recall_at_k": 0.9, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.9629629629629596, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9814814814814815, + "vectors": 71274 + }, + { + "bytes_read_per_query": 6861904.0, + "depth": 50, + "doc_recall_at_k": 0.4394958136856284, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 648, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.3872220880286448, + "p10_recall_at_k": 0.9, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.9785493827160467, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9814814814814815, + "vectors": 71274 + }, + { + "bytes_read_per_query": 6881504.0, + "depth": 100, + "doc_recall_at_k": 0.4394958136856284, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 648, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.3872220880286448, + "p10_recall_at_k": 0.9, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.9787037037037012, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9814814814814815, + "vectors": 71274 + }, + { + "bytes_read_per_query": 6848154.962962963, + "depth": 15, + "doc_recall_at_k": 0.4374467739051073, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 648, + "mean_depth": 14.925925925925926, + "metric": "cosine", + "ndcg_at_k": 0.3864787497670512, + "p10_recall_at_k": 0.8, + "p90_depth": 19, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 648, + "recall_at_k": 0.9341049382716005, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9799382716049383, + "vectors": 71274 + }, + { + "bytes_read_per_query": 6850934.654320988, + "depth": 22, + "doc_recall_at_k": 0.4394958136856284, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 648, + "mean_depth": 22.016975308641975, + "metric": "cosine", + "ndcg_at_k": 0.38725473833484875, + "p10_recall_at_k": 0.9, + "p90_depth": 31, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 648, + "recall_at_k": 0.9711419753086384, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9814814814814815, + "vectors": 71274 + }, + { + "bytes_read_per_query": 6855028.87654321, + "depth": 32, + "doc_recall_at_k": 0.439804455660937, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 648, + "mean_depth": 32.46141975308642, + "metric": "cosine", + "ndcg_at_k": 0.387373383222829, + "p10_recall_at_k": 0.9, + "p90_depth": 50, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 648, + "recall_at_k": 0.9779320987654294, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9814814814814815, + "vectors": 71274 + }, + { + "bytes_read_per_query": 6873024.0, + "depth": 20, + "doc_recall_at_k": 0.43726673275284383, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 648, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.38649561736777843, + "p10_recall_at_k": 0.9, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.9777777777777762, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 71274 + }, + { + "bytes_read_per_query": 6919104.0, + "depth": 50, + "doc_recall_at_k": 0.438503750193565, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 648, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.38702083481577326, + "p10_recall_at_k": 1.0, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.9993827160493826, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 71274 + }, + { + "bytes_read_per_query": 6995904.0, + "depth": 100, + "doc_recall_at_k": 0.438503750193565, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 648, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.38702083481577326, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 1.0, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 71274 + }, + { + "bytes_read_per_query": 6865230.222222222, + "depth": 15, + "doc_recall_at_k": 0.435852123699346, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 648, + "mean_depth": 14.925925925925926, + "metric": "cosine", + "ndcg_at_k": 0.3862409084693553, + "p10_recall_at_k": 0.8, + "p90_depth": 19, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 648, + "recall_at_k": 0.9424382716049347, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9984567901234568, + "vectors": 71274 + }, + { + "bytes_read_per_query": 6876122.074074074, + "depth": 22, + "doc_recall_at_k": 0.4390328507226655, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 648, + "mean_depth": 22.016975308641975, + "metric": "cosine", + "ndcg_at_k": 0.3873170250108132, + "p10_recall_at_k": 0.9, + "p90_depth": 31, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 648, + "recall_at_k": 0.9888888888888874, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 71274 + }, + { + "bytes_read_per_query": 6892164.740740741, + "depth": 32, + "doc_recall_at_k": 0.4388123921688736, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 648, + "mean_depth": 32.46141975308642, + "metric": "cosine", + "ndcg_at_k": 0.3871721300099575, + "p10_recall_at_k": 1.0, + "p90_depth": 50, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 648, + "recall_at_k": 0.9987654320987653, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 71274 + }, + { + "bytes_read_per_query": 6842304.0, + "depth": 0, + "doc_recall_at_k": 0.43593112134778805, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 648, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.3763733494464755, + "p10_recall_at_k": 0.7, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 648, + "recall_at_k": 0.8223765432098744, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.8148148148148148, + "vectors": 71274 + }, + { + "bytes_read_per_query": 6850144.0, + "depth": 20, + "doc_recall_at_k": 0.43957297417945557, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 648, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.38725847048539785, + "p10_recall_at_k": 0.9, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.9689814814814781, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9814814814814815, + "vectors": 71274 + }, + { + "bytes_read_per_query": 6861904.0, + "depth": 50, + "doc_recall_at_k": 0.4394958136856284, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 648, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.3872220880286448, + "p10_recall_at_k": 0.9, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.9787037037037012, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9814814814814815, + "vectors": 71274 + }, + { + "bytes_read_per_query": 6881504.0, + "depth": 100, + "doc_recall_at_k": 0.4394958136856284, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 648, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.3872220880286448, + "p10_recall_at_k": 0.9, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.9787037037037012, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9814814814814815, + "vectors": 71274 + }, + { + "bytes_read_per_query": 6848059.382716049, + "depth": 15, + "doc_recall_at_k": 0.44098758323295356, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 648, + "mean_depth": 14.682098765432098, + "metric": "cosine", + "ndcg_at_k": 0.3876499983063257, + "p10_recall_at_k": 0.9, + "p90_depth": 19, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 648, + "recall_at_k": 0.9373456790123403, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9814814814814815, + "vectors": 71274 + }, + { + "bytes_read_per_query": 6850765.271604938, + "depth": 22, + "doc_recall_at_k": 0.4391871717103198, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 648, + "mean_depth": 21.584876543209877, + "metric": "cosine", + "ndcg_at_k": 0.3870569751700534, + "p10_recall_at_k": 0.9, + "p90_depth": 31, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 648, + "recall_at_k": 0.9733024691357992, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9814814814814815, + "vectors": 71274 + }, + { + "bytes_read_per_query": 6854742.740740741, + "depth": 32, + "doc_recall_at_k": 0.4394958136856284, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 648, + "mean_depth": 31.73148148148148, + "metric": "cosine", + "ndcg_at_k": 0.3872220880286448, + "p10_recall_at_k": 0.9, + "p90_depth": 48, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 648, + "recall_at_k": 0.9780864197530837, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9814814814814815, + "vectors": 71274 + }, + { + "bytes_read_per_query": 6873024.0, + "depth": 20, + "doc_recall_at_k": 0.4384155667720482, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 648, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.3869968056744331, + "p10_recall_at_k": 0.9, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.9842592592592578, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 71274 + }, + { + "bytes_read_per_query": 6919104.0, + "depth": 50, + "doc_recall_at_k": 0.438503750193565, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 648, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.38702083481577326, + "p10_recall_at_k": 1.0, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.9998456790123457, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 71274 + }, + { + "bytes_read_per_query": 6995904.0, + "depth": 100, + "doc_recall_at_k": 0.438503750193565, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 648, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.38702083481577326, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 1.0, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 71274 + }, + { + "bytes_read_per_query": 6864855.703703703, + "depth": 15, + "doc_recall_at_k": 0.4384670071012664, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 648, + "mean_depth": 14.682098765432098, + "metric": "cosine", + "ndcg_at_k": 0.3869970675357867, + "p10_recall_at_k": 0.9, + "p90_depth": 19, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 648, + "recall_at_k": 0.9453703703703659, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 71274 + }, + { + "bytes_read_per_query": 6875458.37037037, + "depth": 22, + "doc_recall_at_k": 0.43819510821825636, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 648, + "mean_depth": 21.584876543209877, + "metric": "cosine", + "ndcg_at_k": 0.38685572195718176, + "p10_recall_at_k": 1.0, + "p90_depth": 31, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 648, + "recall_at_k": 0.9907407407407397, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 71274 + }, + { + "bytes_read_per_query": 6891043.555555556, + "depth": 32, + "doc_recall_at_k": 0.438503750193565, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 648, + "mean_depth": 31.73148148148148, + "metric": "cosine", + "ndcg_at_k": 0.38702083481577326, + "p10_recall_at_k": 1.0, + "p90_depth": 48, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 648, + "recall_at_k": 0.998611111111111, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 71274 + } + ], + "profiles": [ + { + "depth_for_recall": [ + [ + 0.9, + 60 + ], + [ + 0.95, + 105 + ], + [ + 0.99, + 327 + ], + [ + 1.0, + 1720 + ] + ], + "first": "binary", + "first_bytes": 48, + "k": 10, + "median_full_depth": 67.0, + "metric": "cosine", + "neighbour_band": 0.010756641626358032, + "queries": 648, + "rank_noise": 0.01769264290610194, + "recall_at_k": 0.5367283950617282, + "vectors": 71274 + }, + { + "depth_for_recall": [ + [ + 0.9, + 39 + ], + [ + 0.95, + 58 + ], + [ + 0.99, + 143 + ], + [ + 1.0, + 1243 + ] + ], + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "median_full_depth": 42.0, + "metric": "cosine", + "neighbour_band": 0.010756641626358032, + "queries": 648, + "rank_noise": 0.011478020154759652, + "recall_at_k": 0.6001543209876543, + "vectors": 71274 + }, + { + "depth_for_recall": [ + [ + 0.9, + 29 + ], + [ + 0.95, + 44 + ], + [ + 0.99, + 88 + ], + [ + 1.0, + 438 + ] + ], + "first": "pq48", + "first_bytes": 48, + "k": 10, + "median_full_depth": 34.0, + "metric": "cosine", + "neighbour_band": 0.010756641626358032, + "queries": 648, + "rank_noise": 0.010514919675474296, + "recall_at_k": 0.6418209876543218, + "vectors": 71274 + }, + { + "depth_for_recall": [ + [ + 0.9, + 39 + ], + [ + 0.95, + 62 + ], + [ + 0.99, + 157 + ], + [ + 1.0, + 631 + ] + ], + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "median_full_depth": 42.0, + "metric": "cosine", + "neighbour_band": 0.010756641626358032, + "queries": 648, + "rank_noise": 0.011595250017197467, + "recall_at_k": 0.5891975308641977, + "vectors": 71274 + }, + { + "depth_for_recall": [ + [ + 0.9, + 26 + ], + [ + 0.95, + 38 + ], + [ + 0.99, + 71 + ], + [ + 1.0, + 252 + ] + ], + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "median_full_depth": 29.0, + "metric": "cosine", + "neighbour_band": 0.010756641626358032, + "queries": 648, + "rank_noise": 0.01038231096210938, + "recall_at_k": 0.6628086419753089, + "vectors": 71274 + }, + { + "depth_for_recall": [ + [ + 0.9, + 25 + ], + [ + 0.95, + 36 + ], + [ + 0.99, + 75 + ], + [ + 1.0, + 496 + ] + ], + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "median_full_depth": 28.0, + "metric": "cosine", + "neighbour_band": 0.010756641626358032, + "queries": 648, + "rank_noise": 0.010030851916730615, + "recall_at_k": 0.6699074074074076, + "vectors": 71274 + }, + { + "depth_for_recall": [ + [ + 0.9, + 98 + ], + [ + 0.95, + 165 + ], + [ + 0.99, + 453 + ], + [ + 1.0, + 3704 + ] + ], + "first": "pq24", + "first_bytes": 24, + "k": 10, + "median_full_depth": 110.0, + "metric": "cosine", + "neighbour_band": 0.010756641626358032, + "queries": 648, + "rank_noise": 0.014464009981583719, + "recall_at_k": 0.4453703703703707, + "vectors": 71274 + }, + { + "depth_for_recall": [ + [ + 0.9, + 64 + ], + [ + 0.95, + 110 + ], + [ + 0.99, + 275 + ], + [ + 1.0, + 5265 + ] + ], + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "median_full_depth": 70.0, + "metric": "cosine", + "neighbour_band": 0.010756641626358032, + "queries": 648, + "rank_noise": 0.014317320438754092, + "recall_at_k": 0.5061728395061725, + "vectors": 71274 + }, + { + "depth_for_recall": [ + [ + 0.9, + 66 + ], + [ + 0.95, + 111 + ], + [ + 0.99, + 271 + ], + [ + 1.0, + 2206 + ] + ], + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "median_full_depth": 72.0, + "metric": "cosine", + "neighbour_band": 0.010756641626358032, + "queries": 648, + "rank_noise": 0.014069137685018584, + "recall_at_k": 0.4907407407407402, + "vectors": 71274 + }, + { + "depth_for_recall": [ + [ + 0.9, + 13 + ], + [ + 0.95, + 16 + ], + [ + 0.99, + 25 + ], + [ + 1.0, + 65 + ] + ], + "first": "pq96", + "first_bytes": 96, + "k": 10, + "median_full_depth": 15.0, + "metric": "cosine", + "neighbour_band": 0.010756641626358032, + "queries": 648, + "rank_noise": 0.005679709089590111, + "recall_at_k": 0.8165123456790103, + "vectors": 71274 + }, + { + "depth_for_recall": [ + [ + 0.9, + 13 + ], + [ + 0.95, + 16 + ], + [ + 0.99, + 23 + ], + [ + 1.0, + 58 + ] + ], + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "median_full_depth": 14.0, + "metric": "cosine", + "neighbour_band": 0.010756641626358032, + "queries": 648, + "rank_noise": 0.005504157689267154, + "recall_at_k": 0.8223765432098744, + "vectors": 71274 + } + ] +} diff --git a/experiments/2026-11-hybrid-adaptive-matrix/results/fiqa-minilm-k10.json b/experiments/2026-11-hybrid-adaptive-matrix/results/fiqa-minilm-k10.json new file mode 100644 index 0000000..d348422 --- /dev/null +++ b/experiments/2026-11-hybrid-adaptive-matrix/results/fiqa-minilm-k10.json @@ -0,0 +1,3900 @@ +{ + "cells": [ + { + "bytes_read_per_query": 3421152.0, + "depth": 0, + "doc_recall_at_k": 0.4254477604014641, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.3527147190089763, + "p10_recall_at_k": 0.5, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 648, + "recall_at_k": 0.6655864197530874, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.6203703703703703, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3428992.0, + "depth": 20, + "doc_recall_at_k": 0.43638438036586175, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.3646050809255716, + "p10_recall_at_k": 0.7, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.8563271604938248, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9922839506172839, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3440752.0, + "depth": 50, + "doc_recall_at_k": 0.4359238669423855, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.3654117160275599, + "p10_recall_at_k": 0.9, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.9679012345678984, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9953703703703703, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3460352.0, + "depth": 100, + "doc_recall_at_k": 0.4382258216748959, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.36685271755991017, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.989969135802468, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9953703703703703, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3430201.2716049384, + "depth": 23, + "doc_recall_at_k": 0.4406906707832634, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 23.084876543209877, + "metric": "cosine", + "ndcg_at_k": 0.3668756278612038, + "p10_recall_at_k": 0.7, + "p90_depth": 32, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 648, + "recall_at_k": 0.8881172839506142, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9922839506172839, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3441620.0864197533, + "depth": 52, + "doc_recall_at_k": 0.43874022496707693, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 52.214506172839506, + "metric": "cosine", + "ndcg_at_k": 0.366966616362585, + "p10_recall_at_k": 0.9, + "p90_depth": 81, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 648, + "recall_at_k": 0.9765432098765411, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9953703703703703, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3465129.8024691357, + "depth": 112, + "doc_recall_at_k": 0.4382258216748959, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 112.18827160493827, + "metric": "cosine", + "ndcg_at_k": 0.3668305241377324, + "p10_recall_at_k": 1.0, + "p90_depth": 189, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 648, + "recall_at_k": 0.9938271604938267, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9953703703703703, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3451872.0, + "depth": 20, + "doc_recall_at_k": 0.43484117048931864, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.3640474533471147, + "p10_recall_at_k": 0.7, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.856790123456788, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9969135802469136, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3497952.0, + "depth": 50, + "doc_recall_at_k": 0.4361167681769534, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.36542031646367307, + "p10_recall_at_k": 0.9, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.9694444444444417, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3574752.0, + "depth": 100, + "doc_recall_at_k": 0.4389331262016448, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.3670706574611969, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.9932098765432092, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3456610.3703703703, + "depth": 23, + "doc_recall_at_k": 0.4406906707832634, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 23.084876543209877, + "metric": "cosine", + "ndcg_at_k": 0.36676312394130606, + "p10_recall_at_k": 0.7, + "p90_depth": 32, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 648, + "recall_at_k": 0.8885802469135772, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9969135802469136, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3501353.4814814813, + "depth": 52, + "doc_recall_at_k": 0.4389331262016448, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 52.214506172839506, + "metric": "cosine", + "ndcg_at_k": 0.3669838819212713, + "p10_recall_at_k": 0.9, + "p90_depth": 81, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 648, + "recall_at_k": 0.978395061728393, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3593473.185185185, + "depth": 112, + "doc_recall_at_k": 0.4389331262016448, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 112.18827160493827, + "metric": "cosine", + "ndcg_at_k": 0.36704846403901914, + "p10_recall_at_k": 1.0, + "p90_depth": 189, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 648, + "recall_at_k": 0.9970679012345676, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3421152.0, + "depth": 0, + "doc_recall_at_k": 0.41028026757193414, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.35197086253288135, + "p10_recall_at_k": 0.5, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 648, + "recall_at_k": 0.6825617283950618, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.6388888888888888, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3428992.0, + "depth": 20, + "doc_recall_at_k": 0.4343145194997046, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.3641743663917034, + "p10_recall_at_k": 0.7, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.8694444444444412, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9922839506172839, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3440752.0, + "depth": 50, + "doc_recall_at_k": 0.438032920440328, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.3665974130651472, + "p10_recall_at_k": 0.9, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.9733024691358002, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9938271604938271, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3460352.0, + "depth": 100, + "doc_recall_at_k": 0.4362968093292169, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.366169843573182, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.9925925925925919, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9953703703703703, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3430017.975308642, + "depth": 23, + "doc_recall_at_k": 0.4350346841087582, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 22.617283950617285, + "metric": "cosine", + "ndcg_at_k": 0.36498522698712454, + "p10_recall_at_k": 0.8, + "p90_depth": 31, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 648, + "recall_at_k": 0.8983024691357979, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9938271604938271, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3440951.024691358, + "depth": 51, + "doc_recall_at_k": 0.4368112126213979, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 50.507716049382715, + "metric": "cosine", + "ndcg_at_k": 0.36610012539443587, + "p10_recall_at_k": 0.9, + "p90_depth": 78, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 648, + "recall_at_k": 0.979320987654319, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9938271604938271, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3463131.0864197533, + "depth": 107, + "doc_recall_at_k": 0.43668261179835266, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 107.0895061728395, + "metric": "cosine", + "ndcg_at_k": 0.3663844364427974, + "p10_recall_at_k": 1.0, + "p90_depth": 179, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 648, + "recall_at_k": 0.9949074074074068, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9953703703703703, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3451872.0, + "depth": 20, + "doc_recall_at_k": 0.43302851126925196, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.3637940209984614, + "p10_recall_at_k": 0.7, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.8700617283950585, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9969135802469136, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3497952.0, + "depth": 50, + "doc_recall_at_k": 0.43854732373250904, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.36670815290378145, + "p10_recall_at_k": 0.9, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.9757716049382698, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9984567901234568, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3574752.0, + "depth": 100, + "doc_recall_at_k": 0.4370041138559658, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.36638778347446876, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.9959876543209875, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3455892.1481481483, + "depth": 23, + "doc_recall_at_k": 0.4350346841087582, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 22.617283950617285, + "metric": "cosine", + "ndcg_at_k": 0.36490615003205107, + "p10_recall_at_k": 0.8, + "p90_depth": 31, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 648, + "recall_at_k": 0.8990740740740697, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9984567901234568, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3498731.8518518517, + "depth": 51, + "doc_recall_at_k": 0.43751851714814693, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 50.507716049382715, + "metric": "cosine", + "ndcg_at_k": 0.36631806529572264, + "p10_recall_at_k": 0.9, + "p90_depth": 78, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 648, + "recall_at_k": 0.9819444444444428, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9984567901234568, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3585641.4814814813, + "depth": 107, + "doc_recall_at_k": 0.4373899163251016, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 107.0895061728395, + "metric": "cosine", + "ndcg_at_k": 0.36660237634408416, + "p10_recall_at_k": 1.0, + "p90_depth": 179, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 648, + "recall_at_k": 0.9981481481481479, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3421152.0, + "depth": 0, + "doc_recall_at_k": 0.4148987629311702, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.34514311125334995, + "p10_recall_at_k": 0.6, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 648, + "recall_at_k": 0.7162037037037043, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.6712962962962963, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3428992.0, + "depth": 20, + "doc_recall_at_k": 0.43350433431451957, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.36460829357736985, + "p10_recall_at_k": 0.8, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.9035493827160456, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9953703703703703, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3440752.0, + "depth": 50, + "doc_recall_at_k": 0.4345882555373297, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.3653889686962343, + "p10_recall_at_k": 0.9, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.9842592592592576, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9953703703703703, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3460352.0, + "depth": 100, + "doc_recall_at_k": 0.43668261179835266, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.3663844364427974, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.9944444444444437, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9953703703703703, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3429120.8518518517, + "depth": 20, + "doc_recall_at_k": 0.4341859186766594, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 20.328703703703702, + "metric": "cosine", + "ndcg_at_k": 0.36507034647477055, + "p10_recall_at_k": 0.8, + "p90_depth": 28, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 648, + "recall_at_k": 0.9067901234567856, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9953703703703703, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3437334.098765432, + "depth": 41, + "doc_recall_at_k": 0.4353598604756013, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 41.28086419753087, + "metric": "cosine", + "ndcg_at_k": 0.3660880096925949, + "p10_recall_at_k": 0.9, + "p90_depth": 63, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 648, + "recall_at_k": 0.9819444444444424, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9953703703703703, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3452585.1975308643, + "depth": 80, + "doc_recall_at_k": 0.43668261179835266, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 80.18672839506173, + "metric": "cosine", + "ndcg_at_k": 0.3663844364427974, + "p10_recall_at_k": 1.0, + "p90_depth": 135, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 648, + "recall_at_k": 0.9949074074074068, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9953703703703703, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3451872.0, + "depth": 20, + "doc_recall_at_k": 0.43350433431451957, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.3644617724185363, + "p10_recall_at_k": 0.8, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.9041666666666629, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9984567901234568, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3497952.0, + "depth": 50, + "doc_recall_at_k": 0.4349097575949428, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.3654327648608714, + "p10_recall_at_k": 0.9, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.987654320987653, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3574752.0, + "depth": 100, + "doc_recall_at_k": 0.4373899163251016, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.36660237634408416, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.9979938271604936, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3452376.888888889, + "depth": 20, + "doc_recall_at_k": 0.4326427088001163, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 20.328703703703702, + "metric": "cosine", + "ndcg_at_k": 0.36448021023994814, + "p10_recall_at_k": 0.8, + "p90_depth": 28, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 648, + "recall_at_k": 0.9072530864197487, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3484559.4074074076, + "depth": 41, + "doc_recall_at_k": 0.43568136253321454, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 41.28086419753087, + "metric": "cosine", + "ndcg_at_k": 0.36612346166266907, + "p10_recall_at_k": 0.9, + "p90_depth": 63, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 648, + "recall_at_k": 0.9847222222222205, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3544318.814814815, + "depth": 80, + "doc_recall_at_k": 0.4373899163251016, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 80.18672839506173, + "metric": "cosine", + "ndcg_at_k": 0.36660237634408416, + "p10_recall_at_k": 1.0, + "p90_depth": 135, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 648, + "recall_at_k": 0.9984567901234566, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3421152.0, + "depth": 0, + "doc_recall_at_k": 0.40491761325094666, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.34646896786499715, + "p10_recall_at_k": 0.5, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 648, + "recall_at_k": 0.6834876543209883, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.6327160493827161, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3428992.0, + "depth": 20, + "doc_recall_at_k": 0.434277776407406, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.3643961230891485, + "p10_recall_at_k": 0.7, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.8708333333333299, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9938271604938271, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3440752.0, + "depth": 50, + "doc_recall_at_k": 0.43854732373250904, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.3668154176528939, + "p10_recall_at_k": 0.9, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.9737654320987633, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9938271604938271, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3460352.0, + "depth": 100, + "doc_recall_at_k": 0.4362968093292169, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.366169843573182, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.9913580246913571, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9953703703703703, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3430087.5432098764, + "depth": 23, + "doc_recall_at_k": 0.4370151367836552, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 22.794753086419753, + "metric": "cosine", + "ndcg_at_k": 0.36599337768263546, + "p10_recall_at_k": 0.8, + "p90_depth": 31, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 648, + "recall_at_k": 0.8981481481481438, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9938271604938271, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3440927.4320987654, + "depth": 50, + "doc_recall_at_k": 0.4365025706460892, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 50.44753086419753, + "metric": "cosine", + "ndcg_at_k": 0.366074209260929, + "p10_recall_at_k": 0.9, + "p90_depth": 79, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 648, + "recall_at_k": 0.9794753086419733, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9938271604938271, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3463017.358024691, + "depth": 107, + "doc_recall_at_k": 0.43668261179835266, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 106.79938271604938, + "metric": "cosine", + "ndcg_at_k": 0.3663844364427974, + "p10_recall_at_k": 1.0, + "p90_depth": 184, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 648, + "recall_at_k": 0.9947530864197525, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9953703703703703, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3451872.0, + "depth": 20, + "doc_recall_at_k": 0.43299176817695334, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.3640157776959065, + "p10_recall_at_k": 0.7, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.8714506172839471, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9984567901234568, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3497952.0, + "depth": 50, + "doc_recall_at_k": 0.43906172702469004, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.36692615749152807, + "p10_recall_at_k": 0.9, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.9765432098765415, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9984567901234568, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3574752.0, + "depth": 100, + "doc_recall_at_k": 0.4370041138559658, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.36638778347446876, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.9947530864197528, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3456164.7407407407, + "depth": 23, + "doc_recall_at_k": 0.4370151367836552, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 22.794753086419753, + "metric": "cosine", + "ndcg_at_k": 0.3659226449221249, + "p10_recall_at_k": 0.8, + "p90_depth": 31, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 648, + "recall_at_k": 0.8990740740740697, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9984567901234568, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3498639.4074074076, + "depth": 50, + "doc_recall_at_k": 0.4372098751728382, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 50.44753086419753, + "metric": "cosine", + "ndcg_at_k": 0.36629214916221575, + "p10_recall_at_k": 0.9, + "p90_depth": 79, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 648, + "recall_at_k": 0.982098765432097, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9984567901234568, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3585195.8518518517, + "depth": 107, + "doc_recall_at_k": 0.4373899163251016, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 106.79938271604938, + "metric": "cosine", + "ndcg_at_k": 0.36660237634408416, + "p10_recall_at_k": 1.0, + "p90_depth": 184, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 648, + "recall_at_k": 0.9981481481481479, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3421152.0, + "depth": 0, + "doc_recall_at_k": 0.4282059247337025, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.35315903394471326, + "p10_recall_at_k": 0.6, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 648, + "recall_at_k": 0.7455246913580241, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.7021604938271605, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3428992.0, + "depth": 20, + "doc_recall_at_k": 0.4413943010007825, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.3680105725250321, + "p10_recall_at_k": 0.8, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.9268518518518486, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9953703703703703, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3440752.0, + "depth": 50, + "doc_recall_at_k": 0.43732561591357894, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.3663577295888393, + "p10_recall_at_k": 1.0, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.9887345679012333, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9953703703703703, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3460352.0, + "depth": 100, + "doc_recall_at_k": 0.43668261179835266, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.3663844364427974, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.9952160493827154, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9953703703703703, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3428491.111111111, + "depth": 19, + "doc_recall_at_k": 0.44081559729707875, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 18.72222222222222, + "metric": "cosine", + "ndcg_at_k": 0.36763262372500577, + "p10_recall_at_k": 0.8, + "p90_depth": 25, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 648, + "recall_at_k": 0.9206790123456745, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9953703703703703, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3434587.074074074, + "depth": 34, + "doc_recall_at_k": 0.43771141838271477, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 34.273148148148145, + "metric": "cosine", + "ndcg_at_k": 0.36643287011973497, + "p10_recall_at_k": 0.9, + "p90_depth": 50, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 648, + "recall_at_k": 0.9817901234567881, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9953703703703703, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3445354.975308642, + "depth": 62, + "doc_recall_at_k": 0.43668261179835266, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 61.742283950617285, + "metric": "cosine", + "ndcg_at_k": 0.3663844364427974, + "p10_recall_at_k": 1.0, + "p90_depth": 99, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 648, + "recall_at_k": 0.9947530864197526, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9953703703703703, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3451872.0, + "depth": 20, + "doc_recall_at_k": 0.4394652886551035, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.3672778371735497, + "p10_recall_at_k": 0.8, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.9279320987654289, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3497952.0, + "depth": 50, + "doc_recall_at_k": 0.438032920440328, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.36657566949012604, + "p10_recall_at_k": 1.0, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.9916666666666657, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3574752.0, + "depth": 100, + "doc_recall_at_k": 0.4373899163251016, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.36660237634408416, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.9987654320987651, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3449909.3333333335, + "depth": 19, + "doc_recall_at_k": 0.440429794827943, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 18.72222222222222, + "metric": "cosine", + "ndcg_at_k": 0.36734597606845837, + "p10_recall_at_k": 0.8, + "p90_depth": 25, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 648, + "recall_at_k": 0.9217592592592548, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3473795.5555555555, + "depth": 34, + "doc_recall_at_k": 0.43790431961728266, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 34.273148148148145, + "metric": "cosine", + "ndcg_at_k": 0.3664414705558482, + "p10_recall_at_k": 0.9, + "p90_depth": 50, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 648, + "recall_at_k": 0.9842592592592576, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3515988.1481481483, + "depth": 62, + "doc_recall_at_k": 0.4373899163251016, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 61.742283950617285, + "metric": "cosine", + "ndcg_at_k": 0.36660237634408416, + "p10_recall_at_k": 1.0, + "p90_depth": 99, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 648, + "recall_at_k": 0.9981481481481479, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3421152.0, + "depth": 0, + "doc_recall_at_k": 0.41750771091048866, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.3499832627700726, + "p10_recall_at_k": 0.6, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 648, + "recall_at_k": 0.778086419753086, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.7484567901234568, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3428992.0, + "depth": 20, + "doc_recall_at_k": 0.4313977303560637, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.36397570906554083, + "p10_recall_at_k": 0.8, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.9483024691357992, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9953703703703703, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3440752.0, + "depth": 50, + "doc_recall_at_k": 0.4382258216748959, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.36686582329479217, + "p10_recall_at_k": 1.0, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.993055555555555, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9953703703703703, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3460352.0, + "depth": 100, + "doc_recall_at_k": 0.43668261179835266, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.3663844364427974, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.9959876543209871, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9953703703703703, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3427982.358024691, + "depth": 17, + "doc_recall_at_k": 0.43318711895100787, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 17.424382716049383, + "metric": "cosine", + "ndcg_at_k": 0.36515331106324517, + "p10_recall_at_k": 0.8, + "p90_depth": 23, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 648, + "recall_at_k": 0.9316358024691311, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9953703703703703, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3432820.654320988, + "depth": 30, + "doc_recall_at_k": 0.4366826117983526, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 29.766975308641975, + "metric": "cosine", + "ndcg_at_k": 0.3659542360515727, + "p10_recall_at_k": 0.9, + "p90_depth": 44, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 648, + "recall_at_k": 0.9845679012345662, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9953703703703703, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3440941.950617284, + "depth": 50, + "doc_recall_at_k": 0.4371970150905337, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 50.48456790123457, + "metric": "cosine", + "ndcg_at_k": 0.36659377590797093, + "p10_recall_at_k": 1.0, + "p90_depth": 79, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 648, + "recall_at_k": 0.9953703703703698, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9953703703703703, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3451872.0, + "depth": 20, + "doc_recall_at_k": 0.4312048291214958, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.363841902730997, + "p10_recall_at_k": 0.8, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.9499999999999968, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3497952.0, + "depth": 50, + "doc_recall_at_k": 0.4389331262016448, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.3670837631960789, + "p10_recall_at_k": 1.0, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.9962962962962961, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3574752.0, + "depth": 100, + "doc_recall_at_k": 0.4373899163251016, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.36660237634408416, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.9995370370370369, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3447915.8518518517, + "depth": 17, + "doc_recall_at_k": 0.43299421771644003, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 17.424382716049383, + "metric": "cosine", + "ndcg_at_k": 0.36501950472870137, + "p10_recall_at_k": 0.8, + "p90_depth": 23, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 648, + "recall_at_k": 0.9328703703703658, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3466874.074074074, + "depth": 30, + "doc_recall_at_k": 0.4368755130329205, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 29.766975308641975, + "metric": "cosine", + "ndcg_at_k": 0.36596283648768585, + "p10_recall_at_k": 0.9, + "p90_depth": 44, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 648, + "recall_at_k": 0.9870370370370355, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3498696.296296296, + "depth": 50, + "doc_recall_at_k": 0.4373899163251016, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 50.48456790123457, + "metric": "cosine", + "ndcg_at_k": 0.36660237634408416, + "p10_recall_at_k": 1.0, + "p90_depth": 79, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 648, + "recall_at_k": 0.9987654320987653, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 71274 + }, + { + "bytes_read_per_query": 1710576.0, + "depth": 0, + "doc_recall_at_k": 0.3483780879614214, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 648, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.2827961021471452, + "p10_recall_at_k": 0.3, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 648, + "recall_at_k": 0.5069444444444439, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.43209876543209874, + "vectors": 71274 + }, + { + "bytes_read_per_query": 1718416.0, + "depth": 20, + "doc_recall_at_k": 0.39202557559039064, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 648, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.3419938814094672, + "p10_recall_at_k": 0.4, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.6929012345679012, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9320987654320988, + "vectors": 71274 + }, + { + "bytes_read_per_query": 1730176.0, + "depth": 50, + "doc_recall_at_k": 0.4201561812672924, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 648, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.35669768061800894, + "p10_recall_at_k": 0.7, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.8760802469135786, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9814814814814815, + "vectors": 71274 + }, + { + "bytes_read_per_query": 1749776.0, + "depth": 100, + "doc_recall_at_k": 0.4319243813456776, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 648, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.3629208887706261, + "p10_recall_at_k": 0.8, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.9495370370370331, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9907407407407407, + "vectors": 71274 + }, + { + "bytes_read_per_query": 1725764.7901234569, + "depth": 39, + "doc_recall_at_k": 0.41578813419091193, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 648, + "mean_depth": 38.74691358024691, + "metric": "cosine", + "ndcg_at_k": 0.35503889527148014, + "p10_recall_at_k": 0.7, + "p90_depth": 58, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 648, + "recall_at_k": 0.8358024691358007, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9783950617283951, + "vectors": 71274 + }, + { + "bytes_read_per_query": 1760017.6049382717, + "depth": 126, + "doc_recall_at_k": 0.43827726200411393, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 648, + "mean_depth": 126.12654320987654, + "metric": "cosine", + "ndcg_at_k": 0.3667497237480737, + "p10_recall_at_k": 0.9, + "p90_depth": 224, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 648, + "recall_at_k": 0.9655864197530835, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9922839506172839, + "vectors": 71274 + }, + { + "bytes_read_per_query": 1844341.7654320989, + "depth": 341, + "doc_recall_at_k": 0.4386116241440317, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 648, + "mean_depth": 341.2391975308642, + "metric": "cosine", + "ndcg_at_k": 0.3670046678743821, + "p10_recall_at_k": 1.0, + "p90_depth": 685, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 648, + "recall_at_k": 0.9935185185185178, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9953703703703703, + "vectors": 71274 + }, + { + "bytes_read_per_query": 1741296.0, + "depth": 20, + "doc_recall_at_k": 0.3922827772364811, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 648, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.34204410298671056, + "p10_recall_at_k": 0.4, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.6929012345679012, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.933641975308642, + "vectors": 71274 + }, + { + "bytes_read_per_query": 1787376.0, + "depth": 50, + "doc_recall_at_k": 0.4201561812672924, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 648, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.3565805061699114, + "p10_recall_at_k": 0.7, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.8766975308641958, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9861111111111112, + "vectors": 71274 + }, + { + "bytes_read_per_query": 1864176.0, + "depth": 100, + "doc_recall_at_k": 0.4321172825802455, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 648, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.36292948920673934, + "p10_recall_at_k": 0.8, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.9515432098765395, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9953703703703703, + "vectors": 71274 + }, + { + "bytes_read_per_query": 1770091.2592592593, + "depth": 39, + "doc_recall_at_k": 0.41450212596045927, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 648, + "mean_depth": 38.74691358024691, + "metric": "cosine", + "ndcg_at_k": 0.35462728433635915, + "p10_recall_at_k": 0.7, + "p90_depth": 58, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 648, + "recall_at_k": 0.8358024691358007, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9830246913580247, + "vectors": 71274 + }, + { + "bytes_read_per_query": 1904306.3703703703, + "depth": 126, + "doc_recall_at_k": 0.438984566530863, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 648, + "mean_depth": 126.12654320987654, + "metric": "cosine", + "ndcg_at_k": 0.3669676636493604, + "p10_recall_at_k": 0.9, + "p90_depth": 224, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 648, + "recall_at_k": 0.9680555555555529, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9969135802469136, + "vectors": 71274 + }, + { + "bytes_read_per_query": 2234719.4074074076, + "depth": 341, + "doc_recall_at_k": 0.43931892867078065, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 648, + "mean_depth": 341.2391975308642, + "metric": "cosine", + "ndcg_at_k": 0.3672226077756688, + "p10_recall_at_k": 1.0, + "p90_depth": 685, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 648, + "recall_at_k": 0.9970679012345675, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 71274 + }, + { + "bytes_read_per_query": 1710576.0, + "depth": 0, + "doc_recall_at_k": 0.38040310598180976, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 648, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.31040116993194394, + "p10_recall_at_k": 0.4, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 648, + "recall_at_k": 0.5779320987654322, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.5462962962962963, + "vectors": 71274 + }, + { + "bytes_read_per_query": 1718416.0, + "depth": 20, + "doc_recall_at_k": 0.4158406494054642, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 648, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.35523841479543394, + "p10_recall_at_k": 0.5, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.7751543209876532, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9598765432098766, + "vectors": 71274 + }, + { + "bytes_read_per_query": 1730176.0, + "depth": 50, + "doc_recall_at_k": 0.4293829841283545, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 648, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.3625043289547306, + "p10_recall_at_k": 0.8, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.9280864197530841, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9922839506172839, + "vectors": 71274 + }, + { + "bytes_read_per_query": 1749776.0, + "depth": 100, + "doc_recall_at_k": 0.43329857299764724, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 648, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.3650841947986155, + "p10_recall_at_k": 0.9, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.9731481481481462, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9953703703703703, + "vectors": 71274 + }, + { + "bytes_read_per_query": 1722176.9012345679, + "depth": 30, + "doc_recall_at_k": 0.4294454473852622, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 648, + "mean_depth": 29.594135802469136, + "metric": "cosine", + "ndcg_at_k": 0.36217937707084813, + "p10_recall_at_k": 0.7, + "p90_depth": 42, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 648, + "recall_at_k": 0.8574074074074046, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9876543209876543, + "vectors": 71274 + }, + { + "bytes_read_per_query": 1741758.148148148, + "depth": 80, + "doc_recall_at_k": 0.4302121532445608, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 648, + "mean_depth": 79.54629629629629, + "metric": "cosine", + "ndcg_at_k": 0.3635257311944834, + "p10_recall_at_k": 0.9, + "p90_depth": 128, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 648, + "recall_at_k": 0.9668209876543183, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9953703703703703, + "vectors": 71274 + }, + { + "bytes_read_per_query": 1787216.8395061728, + "depth": 196, + "doc_recall_at_k": 0.43591100686008105, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 648, + "mean_depth": 195.51234567901236, + "metric": "cosine", + "ndcg_at_k": 0.36609196570335967, + "p10_recall_at_k": 1.0, + "p90_depth": 339, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 648, + "recall_at_k": 0.9924382716049378, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9953703703703703, + "vectors": 71274 + }, + { + "bytes_read_per_query": 1741296.0, + "depth": 20, + "doc_recall_at_k": 0.4158406494054642, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 648, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.35516497077992126, + "p10_recall_at_k": 0.5, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.7753086419753077, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9645061728395061, + "vectors": 71274 + }, + { + "bytes_read_per_query": 1787376.0, + "depth": 50, + "doc_recall_at_k": 0.42957588536292246, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 648, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.3625057211139804, + "p10_recall_at_k": 0.8, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.9296296296296274, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9969135802469136, + "vectors": 71274 + }, + { + "bytes_read_per_query": 1864176.0, + "depth": 100, + "doc_recall_at_k": 0.43310567176307935, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 648, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.36491865149807906, + "p10_recall_at_k": 0.9, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.9754629629629612, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 71274 + }, + { + "bytes_read_per_query": 1756032.5925925926, + "depth": 30, + "doc_recall_at_k": 0.4296383486198301, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 648, + "mean_depth": 29.594135802469136, + "metric": "cosine", + "ndcg_at_k": 0.362199565939108, + "p10_recall_at_k": 0.7, + "p90_depth": 42, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 648, + "recall_at_k": 0.8580246913580218, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9922839506172839, + "vectors": 71274 + }, + { + "bytes_read_per_query": 1832759.111111111, + "depth": 80, + "doc_recall_at_k": 0.4300192520099929, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 648, + "mean_depth": 79.54629629629629, + "metric": "cosine", + "ndcg_at_k": 0.36336018789394703, + "p10_recall_at_k": 0.9, + "p90_depth": 128, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 648, + "recall_at_k": 0.9689814814814791, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 71274 + }, + { + "bytes_read_per_query": 2010882.962962963, + "depth": 196, + "doc_recall_at_k": 0.43610390809464894, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 648, + "mean_depth": 195.51234567901236, + "metric": "cosine", + "ndcg_at_k": 0.36610056613947284, + "p10_recall_at_k": 1.0, + "p90_depth": 339, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 648, + "recall_at_k": 0.9956790123456787, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 71274 + }, + { + "bytes_read_per_query": 1710576.0, + "depth": 0, + "doc_recall_at_k": 0.3686569517588038, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 648, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.3091279489947749, + "p10_recall_at_k": 0.4, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 648, + "recall_at_k": 0.6001543209876545, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.5416666666666666, + "vectors": 71274 + }, + { + "bytes_read_per_query": 1718416.0, + "depth": 20, + "doc_recall_at_k": 0.40059170938800576, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 648, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.34628723331190076, + "p10_recall_at_k": 0.6, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.7876543209876529, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9722222222222222, + "vectors": 71274 + }, + { + "bytes_read_per_query": 1730176.0, + "depth": 50, + "doc_recall_at_k": 0.43139160650734737, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 648, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.3634323195185981, + "p10_recall_at_k": 0.8, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.9325617283950596, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9922839506172839, + "vectors": 71274 + }, + { + "bytes_read_per_query": 1749776.0, + "depth": 100, + "doc_recall_at_k": 0.43559869057554246, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 648, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.365559208882763, + "p10_recall_at_k": 0.9, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.9746913580246894, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9953703703703703, + "vectors": 71274 + }, + { + "bytes_read_per_query": 1721884.111111111, + "depth": 29, + "doc_recall_at_k": 0.41405805271546026, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 648, + "mean_depth": 28.84722222222222, + "metric": "cosine", + "ndcg_at_k": 0.3543115390117593, + "p10_recall_at_k": 0.7, + "p90_depth": 41, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 648, + "recall_at_k": 0.8604938271604916, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9845679012345679, + "vectors": 71274 + }, + { + "bytes_read_per_query": 1739726.7654320989, + "depth": 74, + "doc_recall_at_k": 0.4357272913985878, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 648, + "mean_depth": 74.3641975308642, + "metric": "cosine", + "ndcg_at_k": 0.3654911108213198, + "p10_recall_at_k": 0.9, + "p90_depth": 117, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 648, + "recall_at_k": 0.9679012345678989, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9953703703703703, + "vectors": 71274 + }, + { + "bytes_read_per_query": 1779278.8395061728, + "depth": 175, + "doc_recall_at_k": 0.4371970150905337, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 648, + "mean_depth": 175.26234567901236, + "metric": "cosine", + "ndcg_at_k": 0.36684803455197923, + "p10_recall_at_k": 1.0, + "p90_depth": 294, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 648, + "recall_at_k": 0.9913580246913571, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9953703703703703, + "vectors": 71274 + }, + { + "bytes_read_per_query": 1741296.0, + "depth": 20, + "doc_recall_at_k": 0.40059170938800576, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 648, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.34620413324535354, + "p10_recall_at_k": 0.6, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.7881172839506159, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9768518518518519, + "vectors": 71274 + }, + { + "bytes_read_per_query": 1787376.0, + "depth": 50, + "doc_recall_at_k": 0.43158450774191526, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 648, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.36344454484118344, + "p10_recall_at_k": 0.8, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.9342592592592573, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9969135802469136, + "vectors": 71274 + }, + { + "bytes_read_per_query": 1864176.0, + "depth": 100, + "doc_recall_at_k": 0.43579159181011035, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 648, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.36556780931887617, + "p10_recall_at_k": 0.9, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.977160493827159, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 71274 + }, + { + "bytes_read_per_query": 1754885.3333333333, + "depth": 29, + "doc_recall_at_k": 0.41405805271546026, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 648, + "mean_depth": 28.84722222222222, + "metric": "cosine", + "ndcg_at_k": 0.3541967019792195, + "p10_recall_at_k": 0.7, + "p90_depth": 41, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 648, + "recall_at_k": 0.8612654320987634, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9891975308641975, + "vectors": 71274 + }, + { + "bytes_read_per_query": 1824799.4074074074, + "depth": 74, + "doc_recall_at_k": 0.4359201926331557, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 648, + "mean_depth": 74.3641975308642, + "metric": "cosine", + "ndcg_at_k": 0.36549971125743297, + "p10_recall_at_k": 0.9, + "p90_depth": 117, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 648, + "recall_at_k": 0.9702160493827139, + "rescore_ms": 1, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 71274 + }, + { + "bytes_read_per_query": 1979778.962962963, + "depth": 175, + "doc_recall_at_k": 0.4373899163251016, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 648, + "mean_depth": 175.26234567901236, + "metric": "cosine", + "ndcg_at_k": 0.3668566349880924, + "p10_recall_at_k": 1.0, + "p90_depth": 294, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 648, + "recall_at_k": 0.9944444444444439, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 71274 + }, + { + "bytes_read_per_query": 6842304.0, + "depth": 0, + "doc_recall_at_k": 0.43741869841406883, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 648, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.3611501871412196, + "p10_recall_at_k": 0.7, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 648, + "recall_at_k": 0.8567901234567868, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.7978395061728395, + "vectors": 71274 + }, + { + "bytes_read_per_query": 6850144.0, + "depth": 20, + "doc_recall_at_k": 0.4383972894389562, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 648, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.36694619645549087, + "p10_recall_at_k": 0.9, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.9844135802469122, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9953703703703703, + "vectors": 71274 + }, + { + "bytes_read_per_query": 6861904.0, + "depth": 50, + "doc_recall_at_k": 0.43668261179835266, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 648, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.3663844364427974, + "p10_recall_at_k": 1.0, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.9962962962962958, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9953703703703703, + "vectors": 71274 + }, + { + "bytes_read_per_query": 6881504.0, + "depth": 100, + "doc_recall_at_k": 0.43668261179835266, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 648, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.3663844364427974, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.9964506172839501, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9953703703703703, + "vectors": 71274 + }, + { + "bytes_read_per_query": 6847788.975308642, + "depth": 14, + "doc_recall_at_k": 0.438738387812462, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 648, + "mean_depth": 13.992283950617283, + "metric": "cosine", + "ndcg_at_k": 0.36689262208005013, + "p10_recall_at_k": 0.9, + "p90_depth": 18, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 648, + "recall_at_k": 0.9549382716049342, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9953703703703703, + "vectors": 71274 + }, + { + "bytes_read_per_query": 6849958.888888889, + "depth": 20, + "doc_recall_at_k": 0.4382258216748959, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 648, + "mean_depth": 19.52777777777778, + "metric": "cosine", + "ndcg_at_k": 0.3668305241377324, + "p10_recall_at_k": 0.9, + "p90_depth": 27, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 648, + "recall_at_k": 0.9887345679012333, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9953703703703703, + "vectors": 71274 + }, + { + "bytes_read_per_query": 6852861.382716049, + "depth": 27, + "doc_recall_at_k": 0.43668261179835266, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 648, + "mean_depth": 26.932098765432098, + "metric": "cosine", + "ndcg_at_k": 0.3663844364427974, + "p10_recall_at_k": 1.0, + "p90_depth": 40, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 648, + "recall_at_k": 0.9953703703703698, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9953703703703703, + "vectors": 71274 + }, + { + "bytes_read_per_query": 6873024.0, + "depth": 20, + "doc_recall_at_k": 0.4391045939657051, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 648, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.36716413635677764, + "p10_recall_at_k": 0.9, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.9874999999999989, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 71274 + }, + { + "bytes_read_per_query": 6919104.0, + "depth": 50, + "doc_recall_at_k": 0.4373899163251016, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 648, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.36660237634408416, + "p10_recall_at_k": 1.0, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.9998456790123457, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 71274 + }, + { + "bytes_read_per_query": 6995904.0, + "depth": 100, + "doc_recall_at_k": 0.4373899163251016, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 648, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.36660237634408416, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 1.0, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 71274 + }, + { + "bytes_read_per_query": 6863796.148148148, + "depth": 14, + "doc_recall_at_k": 0.438738387812462, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 648, + "mean_depth": 13.992283950617283, + "metric": "cosine", + "ndcg_at_k": 0.3667883878889351, + "p10_recall_at_k": 0.9, + "p90_depth": 18, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 648, + "recall_at_k": 0.9563271604938232, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 71274 + }, + { + "bytes_read_per_query": 6872298.666666667, + "depth": 20, + "doc_recall_at_k": 0.4389331262016448, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 648, + "mean_depth": 19.52777777777778, + "metric": "cosine", + "ndcg_at_k": 0.36704846403901914, + "p10_recall_at_k": 1.0, + "p90_depth": 27, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 648, + "recall_at_k": 0.9919753086419744, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 71274 + }, + { + "bytes_read_per_query": 6883671.703703703, + "depth": 27, + "doc_recall_at_k": 0.4373899163251016, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 648, + "mean_depth": 26.932098765432098, + "metric": "cosine", + "ndcg_at_k": 0.36660237634408416, + "p10_recall_at_k": 1.0, + "p90_depth": 40, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 648, + "recall_at_k": 0.9989197530864197, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 71274 + }, + { + "bytes_read_per_query": 6842304.0, + "depth": 0, + "doc_recall_at_k": 0.437953922791886, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 648, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.3682204652185798, + "p10_recall_at_k": 0.8, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 648, + "recall_at_k": 0.9021604938271546, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.8873456790123457, + "vectors": 71274 + }, + { + "bytes_read_per_query": 6850144.0, + "depth": 20, + "doc_recall_at_k": 0.4371970150905337, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 648, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.36659377590797093, + "p10_recall_at_k": 1.0, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.9952160493827155, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9953703703703703, + "vectors": 71274 + }, + { + "bytes_read_per_query": 6861904.0, + "depth": 50, + "doc_recall_at_k": 0.43668261179835266, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 648, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.3663844364427974, + "p10_recall_at_k": 1.0, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.9964506172839501, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9953703703703703, + "vectors": 71274 + }, + { + "bytes_read_per_query": 6881504.0, + "depth": 100, + "doc_recall_at_k": 0.43668261179835266, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 648, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.3663844364427974, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.9964506172839501, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9953703703703703, + "vectors": 71274 + }, + { + "bytes_read_per_query": 6847189.481481481, + "depth": 12, + "doc_recall_at_k": 0.4403716182651368, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 648, + "mean_depth": 12.462962962962964, + "metric": "cosine", + "ndcg_at_k": 0.3676405989034675, + "p10_recall_at_k": 0.9, + "p90_depth": 15, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 648, + "recall_at_k": 0.9677469135802432, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9953703703703703, + "vectors": 71274 + }, + { + "bytes_read_per_query": 6848403.592592592, + "depth": 16, + "doc_recall_at_k": 0.4387916652962951, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 648, + "mean_depth": 15.560185185185185, + "metric": "cosine", + "ndcg_at_k": 0.3671269808443871, + "p10_recall_at_k": 1.0, + "p90_depth": 20, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 648, + "recall_at_k": 0.9915123456790111, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9953703703703703, + "vectors": 71274 + }, + { + "bytes_read_per_query": 6849905.654320988, + "depth": 19, + "doc_recall_at_k": 0.4369912537736614, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 648, + "mean_depth": 19.391975308641975, + "metric": "cosine", + "ndcg_at_k": 0.3665357316369816, + "p10_recall_at_k": 1.0, + "p90_depth": 27, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 648, + "recall_at_k": 0.9959876543209871, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9953703703703703, + "vectors": 71274 + }, + { + "bytes_read_per_query": 6873024.0, + "depth": 20, + "doc_recall_at_k": 0.4373899163251016, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 648, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.36660237634408416, + "p10_recall_at_k": 1.0, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.998611111111111, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 71274 + }, + { + "bytes_read_per_query": 6919104.0, + "depth": 50, + "doc_recall_at_k": 0.4373899163251016, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 648, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.36660237634408416, + "p10_recall_at_k": 1.0, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 1.0, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 71274 + }, + { + "bytes_read_per_query": 6995904.0, + "depth": 100, + "doc_recall_at_k": 0.4373899163251016, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 648, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.36660237634408416, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 1.0, + "rescore_ms": 10, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 71274 + }, + { + "bytes_read_per_query": 6861447.111111111, + "depth": 12, + "doc_recall_at_k": 0.4405645194997047, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 648, + "mean_depth": 12.462962962962964, + "metric": "cosine", + "ndcg_at_k": 0.3676491993395807, + "p10_recall_at_k": 0.9, + "p90_depth": 15, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 648, + "recall_at_k": 0.9697530864197497, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 71274 + }, + { + "bytes_read_per_query": 6866204.444444444, + "depth": 16, + "doc_recall_at_k": 0.438984566530863, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 648, + "mean_depth": 15.560185185185185, + "metric": "cosine", + "ndcg_at_k": 0.3671355812805003, + "p10_recall_at_k": 1.0, + "p90_depth": 20, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 648, + "recall_at_k": 0.994598765432098, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 71274 + }, + { + "bytes_read_per_query": 6872090.074074074, + "depth": 19, + "doc_recall_at_k": 0.4376985583004103, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 648, + "mean_depth": 19.391975308641975, + "metric": "cosine", + "ndcg_at_k": 0.3667536715382684, + "p10_recall_at_k": 1.0, + "p90_depth": 27, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 648, + "recall_at_k": 0.9993827160493828, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 71274 + } + ], + "profiles": [ + { + "depth_for_recall": [ + [ + 0.9, + 26 + ], + [ + 0.95, + 38 + ], + [ + 0.99, + 86 + ], + [ + 1.0, + 539 + ] + ], + "first": "binary", + "first_bytes": 48, + "k": 10, + "median_full_depth": 29.5, + "metric": "cosine", + "neighbour_band": 0.027133822441101074, + "queries": 648, + "rank_noise": 0.025144873356017765, + "recall_at_k": 0.6655864197530874, + "vectors": 71274 + }, + { + "depth_for_recall": [ + [ + 0.9, + 24 + ], + [ + 0.95, + 36 + ], + [ + 0.99, + 73 + ], + [ + 1.0, + 334 + ] + ], + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "median_full_depth": 28.0, + "metric": "cosine", + "neighbour_band": 0.027133822441101074, + "queries": 648, + "rank_noise": 0.023914843104149184, + "recall_at_k": 0.6825617283950618, + "vectors": 71274 + }, + { + "depth_for_recall": [ + [ + 0.9, + 20 + ], + [ + 0.95, + 29 + ], + [ + 0.99, + 56 + ], + [ + 1.0, + 159 + ] + ], + "first": "pq48", + "first_bytes": 48, + "k": 10, + "median_full_depth": 23.0, + "metric": "cosine", + "neighbour_band": 0.027133822441101074, + "queries": 648, + "rank_noise": 0.0219992525810692, + "recall_at_k": 0.7162037037037043, + "vectors": 71274 + }, + { + "depth_for_recall": [ + [ + 0.9, + 24 + ], + [ + 0.95, + 35 + ], + [ + 0.99, + 74 + ], + [ + 1.0, + 383 + ] + ], + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "median_full_depth": 27.0, + "metric": "cosine", + "neighbour_band": 0.027133822441101074, + "queries": 648, + "rank_noise": 0.02402540850726392, + "recall_at_k": 0.6834876543209883, + "vectors": 71274 + }, + { + "depth_for_recall": [ + [ + 0.9, + 18 + ], + [ + 0.95, + 24 + ], + [ + 0.99, + 47 + ], + [ + 1.0, + 190 + ] + ], + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "median_full_depth": 20.0, + "metric": "cosine", + "neighbour_band": 0.027133822441101074, + "queries": 648, + "rank_noise": 0.0202853690269941, + "recall_at_k": 0.7455246913580241, + "vectors": 71274 + }, + { + "depth_for_recall": [ + [ + 0.9, + 16 + ], + [ + 0.95, + 20 + ], + [ + 0.99, + 38 + ], + [ + 1.0, + 550 + ] + ], + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "median_full_depth": 17.0, + "metric": "cosine", + "neighbour_band": 0.027133822441101074, + "queries": 648, + "rank_noise": 0.017908072225576454, + "recall_at_k": 0.778086419753086, + "vectors": 71274 + }, + { + "depth_for_recall": [ + [ + 0.9, + 60 + ], + [ + 0.95, + 99 + ], + [ + 0.99, + 250 + ], + [ + 1.0, + 1106 + ] + ], + "first": "pq24", + "first_bytes": 24, + "k": 10, + "median_full_depth": 67.0, + "metric": "cosine", + "neighbour_band": 0.027133822441101074, + "queries": 648, + "rank_noise": 0.03367902353177581, + "recall_at_k": 0.5069444444444439, + "vectors": 71274 + }, + { + "depth_for_recall": [ + [ + 0.9, + 40 + ], + [ + 0.95, + 63 + ], + [ + 0.99, + 182 + ], + [ + 1.0, + 1411 + ] + ], + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "median_full_depth": 45.0, + "metric": "cosine", + "neighbour_band": 0.027133822441101074, + "queries": 648, + "rank_noise": 0.03128228180662948, + "recall_at_k": 0.5779320987654322, + "vectors": 71274 + }, + { + "depth_for_recall": [ + [ + 0.9, + 37 + ], + [ + 0.95, + 61 + ], + [ + 0.99, + 177 + ], + [ + 1.0, + 6133 + ] + ], + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "median_full_depth": 40.0, + "metric": "cosine", + "neighbour_band": 0.027133822441101074, + "queries": 648, + "rank_noise": 0.030101328131382592, + "recall_at_k": 0.6001543209876545, + "vectors": 71274 + }, + { + "depth_for_recall": [ + [ + 0.9, + 12 + ], + [ + 0.95, + 14 + ], + [ + 0.99, + 22 + ], + [ + 1.0, + 63 + ] + ], + "first": "pq96", + "first_bytes": 96, + "k": 10, + "median_full_depth": 13.0, + "metric": "cosine", + "neighbour_band": 0.027133822441101074, + "queries": 648, + "rank_noise": 0.011738154768883602, + "recall_at_k": 0.8567901234567868, + "vectors": 71274 + }, + { + "depth_for_recall": [ + [ + 0.9, + 10 + ], + [ + 0.95, + 12 + ], + [ + 0.99, + 16 + ], + [ + 1.0, + 23 + ] + ], + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "median_full_depth": 12.0, + "metric": "cosine", + "neighbour_band": 0.027133822441101074, + "queries": 648, + "rank_noise": 0.0076861869961404455, + "recall_at_k": 0.9021604938271546, + "vectors": 71274 + } + ] +} diff --git a/experiments/2026-11-hybrid-adaptive-matrix/results/nfcorpus-bge-small-k10.json b/experiments/2026-11-hybrid-adaptive-matrix/results/nfcorpus-bge-small-k10.json new file mode 100644 index 0000000..1cd4f83 --- /dev/null +++ b/experiments/2026-11-hybrid-adaptive-matrix/results/nfcorpus-bge-small-k10.json @@ -0,0 +1,3900 @@ +{ + "cells": [ + { + "bytes_read_per_query": 300480.0, + "depth": 0, + "doc_recall_at_k": 0.14004003836456763, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.29321160644525546, + "p10_recall_at_k": 0.3, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 323, + "recall_at_k": 0.5560371517027866, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.5356037151702786, + "vectors": 6260 + }, + { + "bytes_read_per_query": 308320.0, + "depth": 20, + "doc_recall_at_k": 0.14555432331135698, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.30985187406128833, + "p10_recall_at_k": 0.4, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.7207430340557278, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9318885448916409, + "vectors": 6260 + }, + { + "bytes_read_per_query": 320080.0, + "depth": 50, + "doc_recall_at_k": 0.14926856083000686, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.31311085569410985, + "p10_recall_at_k": 0.6, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.8640866873065022, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9597523219814241, + "vectors": 6260 + }, + { + "bytes_read_per_query": 339680.0, + "depth": 100, + "doc_recall_at_k": 0.15120477013099584, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.3138913374417529, + "p10_recall_at_k": 0.8, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.923839009287926, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9690402476780186, + "vectors": 6260 + }, + { + "bytes_read_per_query": 310406.21671826625, + "depth": 25, + "doc_recall_at_k": 0.14436285581797528, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 25.321981424148607, + "metric": "cosine", + "ndcg_at_k": 0.3083069702730011, + "p10_recall_at_k": 0.5, + "p90_depth": 38, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 323, + "recall_at_k": 0.7647058823529419, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9566563467492261, + "vectors": 6260 + }, + { + "bytes_read_per_query": 324877.4489164087, + "depth": 62, + "doc_recall_at_k": 0.1511344712152639, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 62.23839009287926, + "metric": "cosine", + "ndcg_at_k": 0.31297658154020386, + "p10_recall_at_k": 0.8, + "p90_depth": 112, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 323, + "recall_at_k": 0.9018575851393198, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9628482972136223, + "vectors": 6260 + }, + { + "bytes_read_per_query": 358140.40866873064, + "depth": 147, + "doc_recall_at_k": 0.15097371256114364, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 147.09287925696594, + "metric": "cosine", + "ndcg_at_k": 0.31460236500927213, + "p10_recall_at_k": 0.9, + "p90_depth": 278, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 323, + "recall_at_k": 0.955727554179567, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.978328173374613, + "vectors": 6260 + }, + { + "bytes_read_per_query": 331200.0, + "depth": 20, + "doc_recall_at_k": 0.14566069345847105, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.3097973045060363, + "p10_recall_at_k": 0.4, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.721981424148607, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9380804953560371, + "vectors": 6260 + }, + { + "bytes_read_per_query": 377280.0, + "depth": 50, + "doc_recall_at_k": 0.1486330024714243, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.31222312359502585, + "p10_recall_at_k": 0.6, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.8702786377708982, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9690402476780186, + "vectors": 6260 + }, + { + "bytes_read_per_query": 454080.0, + "depth": 100, + "doc_recall_at_k": 0.15080401726401932, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.31301204658067466, + "p10_recall_at_k": 0.8, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.9390092879256965, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9845201238390093, + "vectors": 6260 + }, + { + "bytes_read_per_query": 339374.56346749223, + "depth": 25, + "doc_recall_at_k": 0.14442620883320514, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 25.321981424148607, + "metric": "cosine", + "ndcg_at_k": 0.3082259049893465, + "p10_recall_at_k": 0.5, + "p90_depth": 38, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 323, + "recall_at_k": 0.7665634674922607, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9628482972136223, + "vectors": 6260 + }, + { + "bytes_read_per_query": 396078.1671826625, + "depth": 62, + "doc_recall_at_k": 0.1508533034960522, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 62.23839009287926, + "metric": "cosine", + "ndcg_at_k": 0.3126666991289628, + "p10_recall_at_k": 0.8, + "p90_depth": 112, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 323, + "recall_at_k": 0.9108359133126942, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.978328173374613, + "vectors": 6260 + }, + { + "bytes_read_per_query": 526414.6625386997, + "depth": 147, + "doc_recall_at_k": 0.1505729596941671, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 147.09287925696594, + "metric": "cosine", + "ndcg_at_k": 0.3136750692669445, + "p10_recall_at_k": 0.9, + "p90_depth": 278, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 323, + "recall_at_k": 0.9730650154798767, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9938080495356038, + "vectors": 6260 + }, + { + "bytes_read_per_query": 300480.0, + "depth": 0, + "doc_recall_at_k": 0.1417636243417844, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.2875005723745878, + "p10_recall_at_k": 0.3, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 323, + "recall_at_k": 0.5826625386996898, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.5201238390092879, + "vectors": 6260 + }, + { + "bytes_read_per_query": 308320.0, + "depth": 20, + "doc_recall_at_k": 0.14680099183114234, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.3091233032523487, + "p10_recall_at_k": 0.5, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.7582043343653252, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9690402476780186, + "vectors": 6260 + }, + { + "bytes_read_per_query": 320080.0, + "depth": 50, + "doc_recall_at_k": 0.14959766585855885, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.3143036167043981, + "p10_recall_at_k": 0.7, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.902786377708979, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9845201238390093, + "vectors": 6260 + }, + { + "bytes_read_per_query": 339680.0, + "depth": 100, + "doc_recall_at_k": 0.15094027742184882, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.31468157000090535, + "p10_recall_at_k": 0.9, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.9523219814241488, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9845201238390093, + "vectors": 6260 + }, + { + "bytes_read_per_query": 310241.1640866873, + "depth": 25, + "doc_recall_at_k": 0.14838800376891412, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 24.90092879256966, + "metric": "cosine", + "ndcg_at_k": 0.31247673950033755, + "p10_recall_at_k": 0.6, + "p90_depth": 36, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 323, + "recall_at_k": 0.8080495356037161, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.978328173374613, + "vectors": 6260 + }, + { + "bytes_read_per_query": 324635.93808049534, + "depth": 62, + "doc_recall_at_k": 0.1507001875371024, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 61.62229102167183, + "metric": "cosine", + "ndcg_at_k": 0.3139091020764081, + "p10_recall_at_k": 0.8, + "p90_depth": 102, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 323, + "recall_at_k": 0.9334365325077405, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9845201238390093, + "vectors": 6260 + }, + { + "bytes_read_per_query": 358633.13931888546, + "depth": 148, + "doc_recall_at_k": 0.15062135063839766, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 148.3498452012384, + "metric": "cosine", + "ndcg_at_k": 0.3139626881257797, + "p10_recall_at_k": 0.9, + "p90_depth": 268, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 323, + "recall_at_k": 0.9687306501547991, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9845201238390093, + "vectors": 6260 + }, + { + "bytes_read_per_query": 331200.0, + "depth": 20, + "doc_recall_at_k": 0.14668632608180168, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.30856801839852444, + "p10_recall_at_k": 0.5, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.7597523219814243, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9845201238390093, + "vectors": 6260 + }, + { + "bytes_read_per_query": 377280.0, + "depth": 50, + "doc_recall_at_k": 0.14943298226205778, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.3131855439477089, + "p10_recall_at_k": 0.7, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.9120743034055732, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 6260 + }, + { + "bytes_read_per_query": 454080.0, + "depth": 100, + "doc_recall_at_k": 0.15083268580558673, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.3137917810022168, + "p10_recall_at_k": 0.9, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.9693498452012385, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 6260 + }, + { + "bytes_read_per_query": 338727.826625387, + "depth": 25, + "doc_recall_at_k": 0.14806693967076026, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 24.90092879256966, + "metric": "cosine", + "ndcg_at_k": 0.3117876531782273, + "p10_recall_at_k": 0.6, + "p90_depth": 36, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 323, + "recall_at_k": 0.8102167182662545, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9938080495356038, + "vectors": 6260 + }, + { + "bytes_read_per_query": 395131.83900928794, + "depth": 62, + "doc_recall_at_k": 0.15040771602500172, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 61.62229102167183, + "metric": "cosine", + "ndcg_at_k": 0.31281587450116405, + "p10_recall_at_k": 0.8, + "p90_depth": 102, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 323, + "recall_at_k": 0.947368421052632, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 6260 + }, + { + "bytes_read_per_query": 528345.3622291022, + "depth": 148, + "doc_recall_at_k": 0.1502221692231799, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 148.3498452012384, + "metric": "cosine", + "ndcg_at_k": 0.31321784559198507, + "p10_recall_at_k": 1.0, + "p90_depth": 268, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 323, + "recall_at_k": 0.9894736842105266, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 6260 + }, + { + "bytes_read_per_query": 300480.0, + "depth": 0, + "doc_recall_at_k": 0.1450286165688322, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.30503224240862437, + "p10_recall_at_k": 0.5, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 323, + "recall_at_k": 0.6842105263157896, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.6408668730650154, + "vectors": 6260 + }, + { + "bytes_read_per_query": 308320.0, + "depth": 20, + "doc_recall_at_k": 0.1493146320310786, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.3124568936806491, + "p10_recall_at_k": 0.7, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.8613003095975242, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9814241486068112, + "vectors": 6260 + }, + { + "bytes_read_per_query": 320080.0, + "depth": 50, + "doc_recall_at_k": 0.15048929231030617, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.31350941741767663, + "p10_recall_at_k": 0.9, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.9563467492260063, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9845201238390093, + "vectors": 6260 + }, + { + "bytes_read_per_query": 339680.0, + "depth": 100, + "doc_recall_at_k": 0.15103459408569686, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.31458234887825404, + "p10_recall_at_k": 0.9, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.9724458204334364, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9845201238390093, + "vectors": 6260 + }, + { + "bytes_read_per_query": 308309.0773993808, + "depth": 20, + "doc_recall_at_k": 0.1470614682043508, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 19.972136222910216, + "metric": "cosine", + "ndcg_at_k": 0.3100008803465606, + "p10_recall_at_k": 0.7, + "p90_depth": 29, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 323, + "recall_at_k": 0.8730650154798774, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9814241486068112, + "vectors": 6260 + }, + { + "bytes_read_per_query": 316217.040247678, + "depth": 40, + "doc_recall_at_k": 0.14956254475596023, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 40.14551083591331, + "metric": "cosine", + "ndcg_at_k": 0.3135846545702836, + "p10_recall_at_k": 0.9, + "p90_depth": 69, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 323, + "recall_at_k": 0.9520123839009291, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9814241486068112, + "vectors": 6260 + }, + { + "bytes_read_per_query": 331320.5696594427, + "depth": 79, + "doc_recall_at_k": 0.15177136832653215, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 78.6749226006192, + "metric": "cosine", + "ndcg_at_k": 0.3147338263960931, + "p10_recall_at_k": 0.9, + "p90_depth": 145, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 323, + "recall_at_k": 0.9718266253869972, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9845201238390093, + "vectors": 6260 + }, + { + "bytes_read_per_query": 331200.0, + "depth": 20, + "doc_recall_at_k": 0.1490577933145341, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.31153948333094406, + "p10_recall_at_k": 0.7, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.8656346749226017, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9938080495356038, + "vectors": 6260 + }, + { + "bytes_read_per_query": 377280.0, + "depth": 50, + "doc_recall_at_k": 0.15014171048229172, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.3130085789333469, + "p10_recall_at_k": 0.9, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.9721362229102168, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9969040247678018, + "vectors": 6260 + }, + { + "bytes_read_per_query": 454080.0, + "depth": 100, + "doc_recall_at_k": 0.1506354126704791, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.3138340956735098, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.9931888544891639, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 6260 + }, + { + "bytes_read_per_query": 331157.2012383901, + "depth": 20, + "doc_recall_at_k": 0.14687982950355927, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 19.972136222910216, + "metric": "cosine", + "ndcg_at_k": 0.3096876048006169, + "p10_recall_at_k": 0.7, + "p90_depth": 29, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 323, + "recall_at_k": 0.8767801857585154, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9938080495356038, + "vectors": 6260 + }, + { + "bytes_read_per_query": 362143.50464396283, + "depth": 40, + "doc_recall_at_k": 0.14904296430393482, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 40.14551083591331, + "metric": "cosine", + "ndcg_at_k": 0.3127854386587758, + "p10_recall_at_k": 0.9, + "p90_depth": 69, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 323, + "recall_at_k": 0.9668730650154801, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9969040247678018, + "vectors": 6260 + }, + { + "bytes_read_per_query": 421324.6811145511, + "depth": 79, + "doc_recall_at_k": 0.15120018828730344, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 78.6749226006192, + "metric": "cosine", + "ndcg_at_k": 0.31376414565585264, + "p10_recall_at_k": 1.0, + "p90_depth": 145, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 323, + "recall_at_k": 0.9928792569659444, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 6260 + }, + { + "bytes_read_per_query": 300480.0, + "depth": 0, + "doc_recall_at_k": 0.13699985876486354, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.2819200653632115, + "p10_recall_at_k": 0.3, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 323, + "recall_at_k": 0.5835913312693496, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.5139318885448917, + "vectors": 6260 + }, + { + "bytes_read_per_query": 308320.0, + "depth": 20, + "doc_recall_at_k": 0.14544758240868194, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.30876619363343266, + "p10_recall_at_k": 0.5, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.7613003095975236, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9659442724458205, + "vectors": 6260 + }, + { + "bytes_read_per_query": 320080.0, + "depth": 50, + "doc_recall_at_k": 0.14983355916419702, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.314782287026025, + "p10_recall_at_k": 0.7, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.9024767801857594, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9845201238390093, + "vectors": 6260 + }, + { + "bytes_read_per_query": 339680.0, + "depth": 100, + "doc_recall_at_k": 0.15110793994609487, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.31515403708374556, + "p10_recall_at_k": 0.9, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.9541795665634676, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9845201238390093, + "vectors": 6260 + }, + { + "bytes_read_per_query": 310150.14241486066, + "depth": 25, + "doc_recall_at_k": 0.14794895883912282, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 24.6687306501548, + "metric": "cosine", + "ndcg_at_k": 0.31217710250590297, + "p10_recall_at_k": 0.6, + "p90_depth": 36, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 323, + "recall_at_k": 0.806191950464397, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9752321981424149, + "vectors": 6260 + }, + { + "bytes_read_per_query": 324222.092879257, + "depth": 61, + "doc_recall_at_k": 0.15007994131496216, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 60.56656346749226, + "metric": "cosine", + "ndcg_at_k": 0.3137985371521366, + "p10_recall_at_k": 0.8, + "p90_depth": 101, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 323, + "recall_at_k": 0.9297213622291031, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9845201238390093, + "vectors": 6260 + }, + { + "bytes_read_per_query": 357343.0588235294, + "depth": 145, + "doc_recall_at_k": 0.1509939476933802, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 145.05882352941177, + "metric": "cosine", + "ndcg_at_k": 0.3148262802070452, + "p10_recall_at_k": 0.9, + "p90_depth": 257, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 323, + "recall_at_k": 0.9687306501547986, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9845201238390093, + "vectors": 6260 + }, + { + "bytes_read_per_query": 331200.0, + "depth": 20, + "doc_recall_at_k": 0.14503847425963573, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.3078175829318432, + "p10_recall_at_k": 0.5, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.762229102167183, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9814241486068112, + "vectors": 6260 + }, + { + "bytes_read_per_query": 377280.0, + "depth": 50, + "doc_recall_at_k": 0.14966887556769595, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.3136666966866712, + "p10_recall_at_k": 0.7, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.9126934984520131, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 6260 + }, + { + "bytes_read_per_query": 454080.0, + "depth": 100, + "doc_recall_at_k": 0.1509801131975962, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.31406727970054016, + "p10_recall_at_k": 0.9, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.9699690402476783, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 6260 + }, + { + "bytes_read_per_query": 338371.1702786378, + "depth": 25, + "doc_recall_at_k": 0.14781764806165207, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 24.6687306501548, + "metric": "cosine", + "ndcg_at_k": 0.3115463415177824, + "p10_recall_at_k": 0.6, + "p90_depth": 36, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 323, + "recall_at_k": 0.8080495356037157, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9907120743034056, + "vectors": 6260 + }, + { + "bytes_read_per_query": 393510.2414860681, + "depth": 61, + "doc_recall_at_k": 0.14995946842687247, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 60.56656346749226, + "metric": "cosine", + "ndcg_at_k": 0.3128753603067124, + "p10_recall_at_k": 0.8, + "p90_depth": 101, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 323, + "recall_at_k": 0.9436532507739945, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 6260 + }, + { + "bytes_read_per_query": 523290.3529411765, + "depth": 145, + "doc_recall_at_k": 0.15054159523920038, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 145.05882352941177, + "metric": "cosine", + "ndcg_at_k": 0.313697336812831, + "p10_recall_at_k": 1.0, + "p90_depth": 257, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 323, + "recall_at_k": 0.9891640866873065, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 6260 + }, + { + "bytes_read_per_query": 300480.0, + "depth": 0, + "doc_recall_at_k": 0.1434329738452717, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.2955836852829141, + "p10_recall_at_k": 0.4, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 323, + "recall_at_k": 0.6455108359133124, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.6191950464396285, + "vectors": 6260 + }, + { + "bytes_read_per_query": 308320.0, + "depth": 20, + "doc_recall_at_k": 0.14825015542629846, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.3128466986550203, + "p10_recall_at_k": 0.6, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.8297213622291035, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9752321981424149, + "vectors": 6260 + }, + { + "bytes_read_per_query": 320080.0, + "depth": 50, + "doc_recall_at_k": 0.14998361517651623, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.314373305263476, + "p10_recall_at_k": 0.8, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.948916408668731, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9814241486068112, + "vectors": 6260 + }, + { + "bytes_read_per_query": 339680.0, + "depth": 100, + "doc_recall_at_k": 0.15183370568119867, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.3150553571747378, + "p10_recall_at_k": 0.9, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.9705882352941179, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9814241486068112, + "vectors": 6260 + }, + { + "bytes_read_per_query": 308136.74303405575, + "depth": 20, + "doc_recall_at_k": 0.15129197641269868, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 19.53250773993808, + "metric": "cosine", + "ndcg_at_k": 0.31288595845669825, + "p10_recall_at_k": 0.6, + "p90_depth": 27, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 323, + "recall_at_k": 0.8318885448916424, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9752321981424149, + "vectors": 6260 + }, + { + "bytes_read_per_query": 315390.56346749223, + "depth": 38, + "doc_recall_at_k": 0.14990901121848246, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 38.037151702786375, + "metric": "cosine", + "ndcg_at_k": 0.31391326480393456, + "p10_recall_at_k": 0.8, + "p90_depth": 61, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 323, + "recall_at_k": 0.9321981424148613, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9814241486068112, + "vectors": 6260 + }, + { + "bytes_read_per_query": 328827.7894736842, + "depth": 72, + "doc_recall_at_k": 0.15234623305428047, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 72.3157894736842, + "metric": "cosine", + "ndcg_at_k": 0.3151796906015134, + "p10_recall_at_k": 0.9, + "p90_depth": 128, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 323, + "recall_at_k": 0.965325077399381, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9814241486068112, + "vectors": 6260 + }, + { + "bytes_read_per_query": 331200.0, + "depth": 20, + "doc_recall_at_k": 0.1471395020589522, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.3114474192794971, + "p10_recall_at_k": 0.6, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.8328173374613015, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9876160990712074, + "vectors": 6260 + }, + { + "bytes_read_per_query": 377280.0, + "depth": 50, + "doc_recall_at_k": 0.1495312627223364, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.31325344912718245, + "p10_recall_at_k": 0.9, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.9637770897832819, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9969040247678018, + "vectors": 6260 + }, + { + "bytes_read_per_query": 454080.0, + "depth": 100, + "doc_recall_at_k": 0.15140158835925543, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.31411354625642635, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.9919504643962848, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9969040247678018, + "vectors": 6260 + }, + { + "bytes_read_per_query": 330481.9318885449, + "depth": 20, + "doc_recall_at_k": 0.15035171726810517, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 19.53250773993808, + "metric": "cosine", + "ndcg_at_k": 0.3119880567550967, + "p10_recall_at_k": 0.6, + "p90_depth": 27, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 323, + "recall_at_k": 0.8340557275541809, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9876160990712074, + "vectors": 6260 + }, + { + "bytes_read_per_query": 358905.06501547986, + "depth": 38, + "doc_recall_at_k": 0.14877769638120783, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 38.037151702786375, + "metric": "cosine", + "ndcg_at_k": 0.31262934544156884, + "p10_recall_at_k": 0.8, + "p90_depth": 61, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 323, + "recall_at_k": 0.9439628482972142, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9969040247678018, + "vectors": 6260 + }, + { + "bytes_read_per_query": 411557.05263157893, + "depth": 72, + "doc_recall_at_k": 0.15110268692965, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 72.3157894736842, + "metric": "cosine", + "ndcg_at_k": 0.31350622463430516, + "p10_recall_at_k": 0.9, + "p90_depth": 128, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 323, + "recall_at_k": 0.9854489164086687, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9969040247678018, + "vectors": 6260 + }, + { + "bytes_read_per_query": 300480.0, + "depth": 0, + "doc_recall_at_k": 0.14614114742935946, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.2937706795137484, + "p10_recall_at_k": 0.4, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 323, + "recall_at_k": 0.6619195046439622, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.6160990712074303, + "vectors": 6260 + }, + { + "bytes_read_per_query": 308320.0, + "depth": 20, + "doc_recall_at_k": 0.14830324695417854, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.3135229262392862, + "p10_recall_at_k": 0.7, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.8526315789473694, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9690402476780186, + "vectors": 6260 + }, + { + "bytes_read_per_query": 320080.0, + "depth": 50, + "doc_recall_at_k": 0.14942212935436228, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.314001716445157, + "p10_recall_at_k": 0.9, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.9560371517027862, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9845201238390093, + "vectors": 6260 + }, + { + "bytes_read_per_query": 339680.0, + "depth": 100, + "doc_recall_at_k": 0.149048791790297, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.31330522724878135, + "p10_recall_at_k": 0.9, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.9727554179566567, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9845201238390093, + "vectors": 6260 + }, + { + "bytes_read_per_query": 307891.59133126936, + "depth": 19, + "doc_recall_at_k": 0.1471901626340454, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 18.907120743034056, + "metric": "cosine", + "ndcg_at_k": 0.3118830459455606, + "p10_recall_at_k": 0.7, + "p90_depth": 27, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 323, + "recall_at_k": 0.8433436532507759, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9690402476780186, + "vectors": 6260 + }, + { + "bytes_read_per_query": 314425.73374613, + "depth": 36, + "doc_recall_at_k": 0.148199487161488, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 35.575851393188856, + "metric": "cosine", + "ndcg_at_k": 0.31282857787424634, + "p10_recall_at_k": 0.8, + "p90_depth": 57, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 323, + "recall_at_k": 0.93467492260062, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9845201238390093, + "vectors": 6260 + }, + { + "bytes_read_per_query": 326131.12074303406, + "depth": 65, + "doc_recall_at_k": 0.15058507193703063, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 65.43653250773994, + "metric": "cosine", + "ndcg_at_k": 0.31405537106606213, + "p10_recall_at_k": 0.9, + "p90_depth": 117, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 323, + "recall_at_k": 0.9659442724458206, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9845201238390093, + "vectors": 6260 + }, + { + "bytes_read_per_query": 331200.0, + "depth": 20, + "doc_recall_at_k": 0.14843861885938353, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.3128638594049414, + "p10_recall_at_k": 0.7, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.8569659442724464, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9814241486068112, + "vectors": 6260 + }, + { + "bytes_read_per_query": 377280.0, + "depth": 50, + "doc_recall_at_k": 0.1488207114260396, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.31251164761873934, + "p10_recall_at_k": 0.9, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.9708978328173372, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9969040247678018, + "vectors": 6260 + }, + { + "bytes_read_per_query": 454080.0, + "depth": 100, + "doc_recall_at_k": 0.1486166744683538, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.31244676587849435, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.9938080495356039, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9969040247678018, + "vectors": 6260 + }, + { + "bytes_read_per_query": 329521.3374613003, + "depth": 19, + "doc_recall_at_k": 0.14698341575711873, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 18.907120743034056, + "metric": "cosine", + "ndcg_at_k": 0.31126103310156805, + "p10_recall_at_k": 0.7, + "p90_depth": 27, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 323, + "recall_at_k": 0.8461300309597538, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9814241486068112, + "vectors": 6260 + }, + { + "bytes_read_per_query": 355124.5077399381, + "depth": 36, + "doc_recall_at_k": 0.14767606357776403, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 35.575851393188856, + "metric": "cosine", + "ndcg_at_k": 0.3113298658177294, + "p10_recall_at_k": 0.8, + "p90_depth": 57, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 323, + "recall_at_k": 0.9464396284829728, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9969040247678018, + "vectors": 6260 + }, + { + "bytes_read_per_query": 400990.51393188856, + "depth": 65, + "doc_recall_at_k": 0.1502875182444607, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 65.43653250773994, + "metric": "cosine", + "ndcg_at_k": 0.3131417389751787, + "p10_recall_at_k": 0.9, + "p90_depth": 117, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 323, + "recall_at_k": 0.9848297213622291, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 6260 + }, + { + "bytes_read_per_query": 150240.0, + "depth": 0, + "doc_recall_at_k": 0.1324171369609357, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 323, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.2780692723416212, + "p10_recall_at_k": 0.3, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 323, + "recall_at_k": 0.529411764705882, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.4613003095975232, + "vectors": 6260 + }, + { + "bytes_read_per_query": 158080.0, + "depth": 20, + "doc_recall_at_k": 0.13942914952903798, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 323, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.305174423921688, + "p10_recall_at_k": 0.4, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.7024767801857589, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.934984520123839, + "vectors": 6260 + }, + { + "bytes_read_per_query": 169840.0, + "depth": 50, + "doc_recall_at_k": 0.14714524162142112, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 323, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.312078556591918, + "p10_recall_at_k": 0.7, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.8687306501547996, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9721362229102167, + "vectors": 6260 + }, + { + "bytes_read_per_query": 189440.0, + "depth": 100, + "doc_recall_at_k": 0.1506877652343526, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 323, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.31453641872607446, + "p10_recall_at_k": 0.8, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.9315789473684215, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9814241486068112, + "vectors": 6260 + }, + { + "bytes_read_per_query": 163815.57894736843, + "depth": 35, + "doc_recall_at_k": 0.14630812985614544, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 323, + "mean_depth": 34.63157894736842, + "metric": "cosine", + "ndcg_at_k": 0.30917819833158666, + "p10_recall_at_k": 0.6, + "p90_depth": 54, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 323, + "recall_at_k": 0.8160990712074313, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9721362229102167, + "vectors": 6260 + }, + { + "bytes_read_per_query": 194146.42724458204, + "depth": 112, + "doc_recall_at_k": 0.1503235050859674, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 323, + "mean_depth": 112.0061919504644, + "metric": "cosine", + "ndcg_at_k": 0.3150678259944067, + "p10_recall_at_k": 0.8, + "p90_depth": 213, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 323, + "recall_at_k": 0.9414860681114555, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9845201238390093, + "vectors": 6260 + }, + { + "bytes_read_per_query": 271666.55108359136, + "depth": 310, + "doc_recall_at_k": 0.15094077665441813, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 323, + "mean_depth": 309.76160990712077, + "metric": "cosine", + "ndcg_at_k": 0.3143616412301621, + "p10_recall_at_k": 0.9, + "p90_depth": 619, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 323, + "recall_at_k": 0.9708978328173377, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9845201238390093, + "vectors": 6260 + }, + { + "bytes_read_per_query": 180960.0, + "depth": 20, + "doc_recall_at_k": 0.13955207673625142, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 323, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.3049073625201839, + "p10_recall_at_k": 0.4, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.7037151702786381, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9442724458204335, + "vectors": 6260 + }, + { + "bytes_read_per_query": 227040.0, + "depth": 50, + "doc_recall_at_k": 0.147006211853635, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 323, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.31124137284888614, + "p10_recall_at_k": 0.7, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.8764705882352943, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9845201238390093, + "vectors": 6260 + }, + { + "bytes_read_per_query": 303840.0, + "depth": 100, + "doc_recall_at_k": 0.15019931952206614, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 323, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.3134601111901986, + "p10_recall_at_k": 0.8, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.9464396284829723, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9938080495356038, + "vectors": 6260 + }, + { + "bytes_read_per_query": 203434.1052631579, + "depth": 35, + "doc_recall_at_k": 0.14616590044049066, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 323, + "mean_depth": 34.63157894736842, + "metric": "cosine", + "ndcg_at_k": 0.308315044802359, + "p10_recall_at_k": 0.6, + "p90_depth": 54, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 323, + "recall_at_k": 0.8207430340557283, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9845201238390093, + "vectors": 6260 + }, + { + "bytes_read_per_query": 322281.5108359133, + "depth": 112, + "doc_recall_at_k": 0.14973452318811942, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 323, + "mean_depth": 112.0061919504644, + "metric": "cosine", + "ndcg_at_k": 0.3137534779369518, + "p10_recall_at_k": 0.9, + "p90_depth": 213, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 323, + "recall_at_k": 0.9575851393188857, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 6260 + }, + { + "bytes_read_per_query": 626033.8328173375, + "depth": 310, + "doc_recall_at_k": 0.15054159523920038, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 323, + "mean_depth": 309.76160990712077, + "metric": "cosine", + "ndcg_at_k": 0.3136086456460411, + "p10_recall_at_k": 1.0, + "p90_depth": 619, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 323, + "recall_at_k": 0.9925696594427246, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 6260 + }, + { + "bytes_read_per_query": 150240.0, + "depth": 0, + "doc_recall_at_k": 0.12024177369581693, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 323, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.2623027965006762, + "p10_recall_at_k": 0.3, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 323, + "recall_at_k": 0.5130030959752325, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.44891640866873067, + "vectors": 6260 + }, + { + "bytes_read_per_query": 158080.0, + "depth": 20, + "doc_recall_at_k": 0.1258993025739623, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 323, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.2918487171201343, + "p10_recall_at_k": 0.4, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.6922600619195047, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.934984520123839, + "vectors": 6260 + }, + { + "bytes_read_per_query": 169840.0, + "depth": 50, + "doc_recall_at_k": 0.14626962597168938, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 323, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.30930089628973156, + "p10_recall_at_k": 0.6, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.8600619195046451, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9659442724458205, + "vectors": 6260 + }, + { + "bytes_read_per_query": 189440.0, + "depth": 100, + "doc_recall_at_k": 0.1481464179136919, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 323, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.31169389346175835, + "p10_recall_at_k": 0.8, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.9241486068111456, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9814241486068112, + "vectors": 6260 + }, + { + "bytes_read_per_query": 160986.6253869969, + "depth": 27, + "doc_recall_at_k": 0.13610588445631117, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 323, + "mean_depth": 27.41486068111455, + "metric": "cosine", + "ndcg_at_k": 0.30057428809321635, + "p10_recall_at_k": 0.5, + "p90_depth": 40, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 323, + "recall_at_k": 0.758823529411765, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9535603715170279, + "vectors": 6260 + }, + { + "bytes_read_per_query": 178741.91950464397, + "depth": 73, + "doc_recall_at_k": 0.14721106675827875, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 323, + "mean_depth": 72.70897832817337, + "metric": "cosine", + "ndcg_at_k": 0.30995288482151473, + "p10_recall_at_k": 0.7, + "p90_depth": 126, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 323, + "recall_at_k": 0.8978328173374623, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.978328173374613, + "vectors": 6260 + }, + { + "bytes_read_per_query": 218927.38080495357, + "depth": 175, + "doc_recall_at_k": 0.1508099043477998, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 323, + "mean_depth": 175.22291021671828, + "metric": "cosine", + "ndcg_at_k": 0.3140652634978801, + "p10_recall_at_k": 0.9, + "p90_depth": 328, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 323, + "recall_at_k": 0.9538699690402478, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9814241486068112, + "vectors": 6260 + }, + { + "bytes_read_per_query": 180960.0, + "depth": 20, + "doc_recall_at_k": 0.12588457137069578, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 323, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.29162674358823804, + "p10_recall_at_k": 0.4, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.6925696594427246, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9442724458204335, + "vectors": 6260 + }, + { + "bytes_read_per_query": 227040.0, + "depth": 50, + "doc_recall_at_k": 0.14625285050869347, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 323, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.3087285633201344, + "p10_recall_at_k": 0.6, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.8647058823529419, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.978328173374613, + "vectors": 6260 + }, + { + "bytes_read_per_query": 303840.0, + "depth": 100, + "doc_recall_at_k": 0.14792037943847391, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 323, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.3109134649388668, + "p10_recall_at_k": 0.8, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.9371517027863775, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9938080495356038, + "vectors": 6260 + }, + { + "bytes_read_per_query": 192349.22600619195, + "depth": 27, + "doc_recall_at_k": 0.13615492530242837, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 323, + "mean_depth": 27.41486068111455, + "metric": "cosine", + "ndcg_at_k": 0.30047654342305885, + "p10_recall_at_k": 0.5, + "p90_depth": 40, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 323, + "recall_at_k": 0.7594427244582046, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9628482972136223, + "vectors": 6260 + }, + { + "bytes_read_per_query": 261920.9907120743, + "depth": 73, + "doc_recall_at_k": 0.14729095035002282, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 323, + "mean_depth": 72.70897832817337, + "metric": "cosine", + "ndcg_at_k": 0.3100051175802955, + "p10_recall_at_k": 0.7, + "p90_depth": 126, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 323, + "recall_at_k": 0.907739938080496, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9907120743034056, + "vectors": 6260 + }, + { + "bytes_read_per_query": 419382.3900928793, + "depth": 175, + "doc_recall_at_k": 0.15035755189361996, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 323, + "mean_depth": 175.22291021671828, + "metric": "cosine", + "ndcg_at_k": 0.3129697584461846, + "p10_recall_at_k": 0.9, + "p90_depth": 328, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 323, + "recall_at_k": 0.9718266253869968, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9969040247678018, + "vectors": 6260 + }, + { + "bytes_read_per_query": 150240.0, + "depth": 0, + "doc_recall_at_k": 0.11428329657652923, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 323, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.2544798169380498, + "p10_recall_at_k": 0.3, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 323, + "recall_at_k": 0.5130030959752322, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.4551083591331269, + "vectors": 6260 + }, + { + "bytes_read_per_query": 158080.0, + "depth": 20, + "doc_recall_at_k": 0.12943811003161726, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 323, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.2913414590968704, + "p10_recall_at_k": 0.4, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.6931888544891647, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9287925696594427, + "vectors": 6260 + }, + { + "bytes_read_per_query": 169840.0, + "depth": 50, + "doc_recall_at_k": 0.14355797483112867, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 323, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.30652281216142213, + "p10_recall_at_k": 0.7, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.8640866873065026, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9690402476780186, + "vectors": 6260 + }, + { + "bytes_read_per_query": 189440.0, + "depth": 100, + "doc_recall_at_k": 0.14804873166337715, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 323, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.31205816643499085, + "p10_recall_at_k": 0.8, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.9266253869969044, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.978328173374613, + "vectors": 6260 + }, + { + "bytes_read_per_query": 161398.04334365326, + "depth": 28, + "doc_recall_at_k": 0.14013434791476978, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 323, + "mean_depth": 28.46439628482972, + "metric": "cosine", + "ndcg_at_k": 0.30348960653653373, + "p10_recall_at_k": 0.5, + "p90_depth": 42, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 323, + "recall_at_k": 0.7659442724458212, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9535603715170279, + "vectors": 6260 + }, + { + "bytes_read_per_query": 179680.04953560373, + "depth": 75, + "doc_recall_at_k": 0.14747883313079274, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 323, + "mean_depth": 75.10216718266254, + "metric": "cosine", + "ndcg_at_k": 0.31078503460289897, + "p10_recall_at_k": 0.8, + "p90_depth": 136, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 323, + "recall_at_k": 0.9058823529411772, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9752321981424149, + "vectors": 6260 + }, + { + "bytes_read_per_query": 220779.36842105264, + "depth": 180, + "doc_recall_at_k": 0.14913069589696892, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 323, + "mean_depth": 179.94736842105263, + "metric": "cosine", + "ndcg_at_k": 0.3133589969907148, + "p10_recall_at_k": 0.9, + "p90_depth": 349, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 323, + "recall_at_k": 0.9554179566563467, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9814241486068112, + "vectors": 6260 + }, + { + "bytes_read_per_query": 180960.0, + "depth": 20, + "doc_recall_at_k": 0.1283995516268203, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 323, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.2898740856015263, + "p10_recall_at_k": 0.4, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.6938080495356043, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9380804953560371, + "vectors": 6260 + }, + { + "bytes_read_per_query": 227040.0, + "depth": 50, + "doc_recall_at_k": 0.1438361273534386, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 323, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.30616908171542584, + "p10_recall_at_k": 0.7, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.8715170278637779, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9814241486068112, + "vectors": 6260 + }, + { + "bytes_read_per_query": 303840.0, + "depth": 100, + "doc_recall_at_k": 0.1476479787964006, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 323, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.3112182021980953, + "p10_recall_at_k": 0.8, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.9405572755417957, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9907120743034056, + "vectors": 6260 + }, + { + "bytes_read_per_query": 193961.31269349845, + "depth": 28, + "doc_recall_at_k": 0.13953152125699167, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 323, + "mean_depth": 28.46439628482972, + "metric": "cosine", + "ndcg_at_k": 0.30297347901243343, + "p10_recall_at_k": 0.5, + "p90_depth": 42, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 323, + "recall_at_k": 0.7684210526315796, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9628482972136223, + "vectors": 6260 + }, + { + "bytes_read_per_query": 265596.92879256967, + "depth": 75, + "doc_recall_at_k": 0.14753584456508856, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 323, + "mean_depth": 75.10216718266254, + "metric": "cosine", + "ndcg_at_k": 0.31053965102786735, + "p10_recall_at_k": 0.8, + "p90_depth": 136, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 323, + "recall_at_k": 0.9167182662538705, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9907120743034056, + "vectors": 6260 + }, + { + "bytes_read_per_query": 426639.15789473685, + "depth": 180, + "doc_recall_at_k": 0.1486985785750257, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 323, + "mean_depth": 179.94736842105263, + "metric": "cosine", + "ndcg_at_k": 0.3124273759408907, + "p10_recall_at_k": 0.9, + "p90_depth": 349, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 323, + "recall_at_k": 0.9739938080495355, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9969040247678018, + "vectors": 6260 + }, + { + "bytes_read_per_query": 600960.0, + "depth": 0, + "doc_recall_at_k": 0.14798313755819867, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 323, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.31244343408405784, + "p10_recall_at_k": 0.7, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 323, + "recall_at_k": 0.8349845201238411, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.7956656346749226, + "vectors": 6260 + }, + { + "bytes_read_per_query": 608800.0, + "depth": 20, + "doc_recall_at_k": 0.15033572161362085, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 323, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.3141140313730846, + "p10_recall_at_k": 0.9, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.9625386996904027, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9845201238390093, + "vectors": 6260 + }, + { + "bytes_read_per_query": 620560.0, + "depth": 50, + "doc_recall_at_k": 0.15094077665441813, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 323, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.3143486384401113, + "p10_recall_at_k": 0.9, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.9764705882352942, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9845201238390093, + "vectors": 6260 + }, + { + "bytes_read_per_query": 640160.0, + "depth": 100, + "doc_recall_at_k": 0.15094077665441813, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 323, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.3143486384401113, + "p10_recall_at_k": 0.9, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.9767801857585141, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9845201238390093, + "vectors": 6260 + }, + { + "bytes_read_per_query": 606342.4148606812, + "depth": 14, + "doc_recall_at_k": 0.14893906992173178, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 323, + "mean_depth": 13.730650154798761, + "metric": "cosine", + "ndcg_at_k": 0.3136826053498692, + "p10_recall_at_k": 0.8, + "p90_depth": 17, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 323, + "recall_at_k": 0.9238390092879267, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9845201238390093, + "vectors": 6260 + }, + { + "bytes_read_per_query": 608434.6996904025, + "depth": 19, + "doc_recall_at_k": 0.14875036321263133, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 323, + "mean_depth": 19.068111455108358, + "metric": "cosine", + "ndcg_at_k": 0.3138775747540555, + "p10_recall_at_k": 0.9, + "p90_depth": 27, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 323, + "recall_at_k": 0.9671826625386997, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9845201238390093, + "vectors": 6260 + }, + { + "bytes_read_per_query": 611286.7120743034, + "depth": 26, + "doc_recall_at_k": 0.1511228928445474, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 323, + "mean_depth": 26.343653250773993, + "metric": "cosine", + "ndcg_at_k": 0.3145456068246281, + "p10_recall_at_k": 0.9, + "p90_depth": 40, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 323, + "recall_at_k": 0.9739938080495355, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9845201238390093, + "vectors": 6260 + }, + { + "bytes_read_per_query": 631680.0, + "depth": 20, + "doc_recall_at_k": 0.1499507205607601, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 323, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.3133732113312625, + "p10_recall_at_k": 0.9, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.9780185758513933, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 6260 + }, + { + "bytes_read_per_query": 677760.0, + "depth": 50, + "doc_recall_at_k": 0.15054159523920038, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 323, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.31359564285599023, + "p10_recall_at_k": 1.0, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.9996904024767801, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 6260 + }, + { + "bytes_read_per_query": 754560.0, + "depth": 100, + "doc_recall_at_k": 0.15054159523920038, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 323, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.31359564285599023, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 1.0, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 6260 + }, + { + "bytes_read_per_query": 622050.2786377709, + "depth": 14, + "doc_recall_at_k": 0.14878191857268913, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 323, + "mean_depth": 13.730650154798761, + "metric": "cosine", + "ndcg_at_k": 0.31304872103981957, + "p10_recall_at_k": 0.8, + "p90_depth": 17, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 323, + "recall_at_k": 0.9312693498452022, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 6260 + }, + { + "bytes_read_per_query": 630248.6191950465, + "depth": 19, + "doc_recall_at_k": 0.1487351592013942, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 323, + "mean_depth": 19.068111455108358, + "metric": "cosine", + "ndcg_at_k": 0.31341222551181197, + "p10_recall_at_k": 0.9, + "p90_depth": 27, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 323, + "recall_at_k": 0.9845201238390089, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 6260 + }, + { + "bytes_read_per_query": 641423.8513931888, + "depth": 26, + "doc_recall_at_k": 0.15072371142932967, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 323, + "mean_depth": 26.343653250773993, + "metric": "cosine", + "ndcg_at_k": 0.3137926112405071, + "p10_recall_at_k": 1.0, + "p90_depth": 40, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 323, + "recall_at_k": 0.9965944272445818, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 6260 + }, + { + "bytes_read_per_query": 600960.0, + "depth": 0, + "doc_recall_at_k": 0.1494171144144571, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 323, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.3099658330753379, + "p10_recall_at_k": 0.6, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 323, + "recall_at_k": 0.810526315789475, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.7956656346749226, + "vectors": 6260 + }, + { + "bytes_read_per_query": 608800.0, + "depth": 20, + "doc_recall_at_k": 0.1499057738891321, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 323, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.31382765945309055, + "p10_recall_at_k": 0.9, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.9563467492260059, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9845201238390093, + "vectors": 6260 + }, + { + "bytes_read_per_query": 620560.0, + "depth": 50, + "doc_recall_at_k": 0.15094077665441813, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 323, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.3143486384401113, + "p10_recall_at_k": 0.9, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.9767801857585141, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9845201238390093, + "vectors": 6260 + }, + { + "bytes_read_per_query": 640160.0, + "depth": 100, + "doc_recall_at_k": 0.15094077665441813, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 323, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.3143486384401113, + "p10_recall_at_k": 0.9, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.9767801857585141, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9845201238390093, + "vectors": 6260 + }, + { + "bytes_read_per_query": 606456.4953560372, + "depth": 14, + "doc_recall_at_k": 0.15018889990145595, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 323, + "mean_depth": 14.021671826625386, + "metric": "cosine", + "ndcg_at_k": 0.31436008950291255, + "p10_recall_at_k": 0.8, + "p90_depth": 18, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 323, + "recall_at_k": 0.9160990712074308, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9845201238390093, + "vectors": 6260 + }, + { + "bytes_read_per_query": 608695.6284829722, + "depth": 20, + "doc_recall_at_k": 0.1519423116825708, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 323, + "mean_depth": 19.73374613003096, + "metric": "cosine", + "ndcg_at_k": 0.3152283428129784, + "p10_recall_at_k": 0.9, + "p90_depth": 28, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 323, + "recall_at_k": 0.9616099071207432, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9845201238390093, + "vectors": 6260 + }, + { + "bytes_read_per_query": 611840.1238390093, + "depth": 28, + "doc_recall_at_k": 0.151460448276791, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 323, + "mean_depth": 27.75541795665635, + "metric": "cosine", + "ndcg_at_k": 0.3146112956501607, + "p10_recall_at_k": 0.9, + "p90_depth": 44, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 323, + "recall_at_k": 0.9736842105263159, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9845201238390093, + "vectors": 6260 + }, + { + "bytes_read_per_query": 631680.0, + "depth": 20, + "doc_recall_at_k": 0.1494339498926114, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 323, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.3124772885233802, + "p10_recall_at_k": 0.9, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.9693498452012381, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 6260 + }, + { + "bytes_read_per_query": 677760.0, + "depth": 50, + "doc_recall_at_k": 0.15054159523920038, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 323, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.31359564285599023, + "p10_recall_at_k": 1.0, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.9996904024767801, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 6260 + }, + { + "bytes_read_per_query": 754560.0, + "depth": 100, + "doc_recall_at_k": 0.15054159523920038, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 323, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.31359564285599023, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 1.0, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 6260 + }, + { + "bytes_read_per_query": 622497.2879256966, + "depth": 14, + "doc_recall_at_k": 0.14948318774108987, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 323, + "mean_depth": 14.021671826625386, + "metric": "cosine", + "ndcg_at_k": 0.3129086004037166, + "p10_recall_at_k": 0.8, + "p90_depth": 18, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 323, + "recall_at_k": 0.9210526315789477, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 6260 + }, + { + "bytes_read_per_query": 631271.0340557275, + "depth": 20, + "doc_recall_at_k": 0.15145632309348447, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 323, + "mean_depth": 19.73374613003096, + "metric": "cosine", + "ndcg_at_k": 0.31387628902498027, + "p10_recall_at_k": 0.9, + "p90_depth": 28, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 323, + "recall_at_k": 0.9758513931888546, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 6260 + }, + { + "bytes_read_per_query": 643592.3219814241, + "depth": 28, + "doc_recall_at_k": 0.15100885941250689, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 323, + "mean_depth": 27.75541795665635, + "metric": "cosine", + "ndcg_at_k": 0.3134643632970059, + "p10_recall_at_k": 1.0, + "p90_depth": 44, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 323, + "recall_at_k": 0.9941176470588238, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 6260 + } + ], + "profiles": [ + { + "depth_for_recall": [ + [ + 0.9, + 64 + ], + [ + 0.95, + 116 + ], + [ + 0.99, + 424 + ], + [ + 1.0, + 1536 + ] + ], + "first": "binary", + "first_bytes": 48, + "k": 10, + "median_full_depth": 58.0, + "metric": "cosine", + "neighbour_band": 0.012125968933105469, + "queries": 323, + "rank_noise": 0.02005583883466805, + "recall_at_k": 0.5560371517027866, + "vectors": 6260 + }, + { + "depth_for_recall": [ + [ + 0.9, + 46 + ], + [ + 0.95, + 74 + ], + [ + 0.99, + 170 + ], + [ + 1.0, + 698 + ] + ], + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "median_full_depth": 42.0, + "metric": "cosine", + "neighbour_band": 0.012125968933105469, + "queries": 323, + "rank_noise": 0.013492521855551734, + "recall_at_k": 0.5826625386996898, + "vectors": 6260 + }, + { + "depth_for_recall": [ + [ + 0.9, + 25 + ], + [ + 0.95, + 37 + ], + [ + 0.99, + 86 + ], + [ + 1.0, + 262 + ] + ], + "first": "pq48", + "first_bytes": 48, + "k": 10, + "median_full_depth": 25.0, + "metric": "cosine", + "neighbour_band": 0.012125968933105469, + "queries": 323, + "rank_noise": 0.010998554937267174, + "recall_at_k": 0.6842105263157896, + "vectors": 6260 + }, + { + "depth_for_recall": [ + [ + 0.9, + 46 + ], + [ + 0.95, + 74 + ], + [ + 0.99, + 181 + ], + [ + 1.0, + 714 + ] + ], + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "median_full_depth": 42.0, + "metric": "cosine", + "neighbour_band": 0.012125968933105469, + "queries": 323, + "rank_noise": 0.013678202417480445, + "recall_at_k": 0.5835913312693496, + "vectors": 6260 + }, + { + "depth_for_recall": [ + [ + 0.9, + 29 + ], + [ + 0.95, + 43 + ], + [ + 0.99, + 93 + ], + [ + 1.0, + 259 + ] + ], + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "median_full_depth": 29.0, + "metric": "cosine", + "neighbour_band": 0.012125968933105469, + "queries": 323, + "rank_noise": 0.012481752227925458, + "recall_at_k": 0.6455108359133124, + "vectors": 6260 + }, + { + "depth_for_recall": [ + [ + 0.9, + 26 + ], + [ + 0.95, + 39 + ], + [ + 0.99, + 80 + ], + [ + 1.0, + 363 + ] + ], + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "median_full_depth": 26.0, + "metric": "cosine", + "neighbour_band": 0.012125968933105469, + "queries": 323, + "rank_noise": 0.01203099135973376, + "recall_at_k": 0.6619195046439622, + "vectors": 6260 + }, + { + "depth_for_recall": [ + [ + 0.9, + 63 + ], + [ + 0.95, + 106 + ], + [ + 0.99, + 315 + ], + [ + 1.0, + 1091 + ] + ], + "first": "pq24", + "first_bytes": 24, + "k": 10, + "median_full_depth": 62.0, + "metric": "cosine", + "neighbour_band": 0.012125968933105469, + "queries": 323, + "rank_noise": 0.015120641347280507, + "recall_at_k": 0.529411764705882, + "vectors": 6260 + }, + { + "depth_for_recall": [ + [ + 0.9, + 66 + ], + [ + 0.95, + 120 + ], + [ + 0.99, + 321 + ], + [ + 1.0, + 1324 + ] + ], + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "median_full_depth": 64.0, + "metric": "cosine", + "neighbour_band": 0.012125968933105469, + "queries": 323, + "rank_noise": 0.016914241226177223, + "recall_at_k": 0.5130030959752325, + "vectors": 6260 + }, + { + "depth_for_recall": [ + [ + 0.9, + 64 + ], + [ + 0.95, + 112 + ], + [ + 0.99, + 305 + ], + [ + 1.0, + 992 + ] + ], + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "median_full_depth": 60.0, + "metric": "cosine", + "neighbour_band": 0.012125968933105469, + "queries": 323, + "rank_noise": 0.01697989232996658, + "recall_at_k": 0.5130030959752322, + "vectors": 6260 + }, + { + "depth_for_recall": [ + [ + 0.9, + 13 + ], + [ + 0.95, + 16 + ], + [ + 0.99, + 25 + ], + [ + 1.0, + 72 + ] + ], + "first": "pq96", + "first_bytes": 96, + "k": 10, + "median_full_depth": 14.0, + "metric": "cosine", + "neighbour_band": 0.012125968933105469, + "queries": 323, + "rank_noise": 0.006044045887215805, + "recall_at_k": 0.8349845201238411, + "vectors": 6260 + }, + { + "depth_for_recall": [ + [ + 0.9, + 14 + ], + [ + 0.95, + 17 + ], + [ + 0.99, + 29 + ], + [ + 1.0, + 55 + ] + ], + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "median_full_depth": 14.0, + "metric": "cosine", + "neighbour_band": 0.012125968933105469, + "queries": 323, + "rank_noise": 0.006860505196857978, + "recall_at_k": 0.810526315789475, + "vectors": 6260 + } + ] +} diff --git a/experiments/2026-11-hybrid-adaptive-matrix/results/nfcorpus-minilm-k10.json b/experiments/2026-11-hybrid-adaptive-matrix/results/nfcorpus-minilm-k10.json new file mode 100644 index 0000000..7b2a447 --- /dev/null +++ b/experiments/2026-11-hybrid-adaptive-matrix/results/nfcorpus-minilm-k10.json @@ -0,0 +1,3900 @@ +{ + "cells": [ + { + "bytes_read_per_query": 300480.0, + "depth": 0, + "doc_recall_at_k": 0.1323602119067216, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.27221531371670776, + "p10_recall_at_k": 0.5, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 323, + "recall_at_k": 0.6690402476780186, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.6470588235294118, + "vectors": 6260 + }, + { + "bytes_read_per_query": 308320.0, + "depth": 20, + "doc_recall_at_k": 0.13394676509996806, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.2796031413328297, + "p10_recall_at_k": 0.7, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.8513931888544893, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9845201238390093, + "vectors": 6260 + }, + { + "bytes_read_per_query": 320080.0, + "depth": 50, + "doc_recall_at_k": 0.13587406923511589, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.2827704687548993, + "p10_recall_at_k": 0.9, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.9594427244582038, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9876160990712074, + "vectors": 6260 + }, + { + "bytes_read_per_query": 339680.0, + "depth": 100, + "doc_recall_at_k": 0.13540153949587275, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.2819581504344227, + "p10_recall_at_k": 0.9, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.9866873065015479, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9938080495356038, + "vectors": 6260 + }, + { + "bytes_read_per_query": 308915.8885448916, + "depth": 22, + "doc_recall_at_k": 0.1340780041277946, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 21.520123839009287, + "metric": "cosine", + "ndcg_at_k": 0.2791837041338206, + "p10_recall_at_k": 0.7, + "p90_depth": 31, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 323, + "recall_at_k": 0.8622291021671832, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9876160990712074, + "vectors": 6260 + }, + { + "bytes_read_per_query": 319198.9102167183, + "depth": 48, + "doc_recall_at_k": 0.13566649446922888, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 47.752321981424146, + "metric": "cosine", + "ndcg_at_k": 0.2819110030451396, + "p10_recall_at_k": 0.9, + "p90_depth": 77, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 323, + "recall_at_k": 0.9690402476780184, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9907120743034056, + "vectors": 6260 + }, + { + "bytes_read_per_query": 340988.2848297214, + "depth": 103, + "doc_recall_at_k": 0.13540153949587275, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 103.3374613003096, + "metric": "cosine", + "ndcg_at_k": 0.2819372685500173, + "p10_recall_at_k": 1.0, + "p90_depth": 190, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 323, + "recall_at_k": 0.9922600619195044, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9938080495356038, + "vectors": 6260 + }, + { + "bytes_read_per_query": 331200.0, + "depth": 20, + "doc_recall_at_k": 0.1331082718079144, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.27922321371175185, + "p10_recall_at_k": 0.7, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.8517027863777092, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9907120743034056, + "vectors": 6260 + }, + { + "bytes_read_per_query": 377280.0, + "depth": 50, + "doc_recall_at_k": 0.13751437586089263, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.28315264636433146, + "p10_recall_at_k": 0.9, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.9619195046439623, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9938080495356038, + "vectors": 6260 + }, + { + "bytes_read_per_query": 454080.0, + "depth": 100, + "doc_recall_at_k": 0.13698846723833571, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.28216709892291303, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.9904024767801856, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 6260 + }, + { + "bytes_read_per_query": 333534.9102167183, + "depth": 22, + "doc_recall_at_k": 0.13273641486050877, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 21.520123839009287, + "metric": "cosine", + "ndcg_at_k": 0.2784196756523232, + "p10_recall_at_k": 0.7, + "p90_depth": 31, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 323, + "recall_at_k": 0.8622291021671832, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9938080495356038, + "vectors": 6260 + }, + { + "bytes_read_per_query": 373827.5665634675, + "depth": 48, + "doc_recall_at_k": 0.13725686601061532, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 47.752321981424146, + "metric": "cosine", + "ndcg_at_k": 0.2820962122700549, + "p10_recall_at_k": 0.9, + "p90_depth": 77, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 323, + "recall_at_k": 0.9715170278637768, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9969040247678018, + "vectors": 6260 + }, + { + "bytes_read_per_query": 459206.3405572756, + "depth": 103, + "doc_recall_at_k": 0.13698846723833571, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 103.3374613003096, + "metric": "cosine", + "ndcg_at_k": 0.2821462170385076, + "p10_recall_at_k": 1.0, + "p90_depth": 190, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 323, + "recall_at_k": 0.9956656346749223, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 6260 + }, + { + "bytes_read_per_query": 300480.0, + "depth": 0, + "doc_recall_at_k": 0.13227598658615683, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.2776443014277795, + "p10_recall_at_k": 0.5, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 323, + "recall_at_k": 0.6907120743034053, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.718266253869969, + "vectors": 6260 + }, + { + "bytes_read_per_query": 308320.0, + "depth": 20, + "doc_recall_at_k": 0.13500278125678464, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.28305081519379827, + "p10_recall_at_k": 0.7, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.8681114551083603, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9876160990712074, + "vectors": 6260 + }, + { + "bytes_read_per_query": 320080.0, + "depth": 50, + "doc_recall_at_k": 0.13612118155867187, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.2812542357472304, + "p10_recall_at_k": 0.9, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.962538699690403, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9907120743034056, + "vectors": 6260 + }, + { + "bytes_read_per_query": 339680.0, + "depth": 100, + "doc_recall_at_k": 0.13649672723750964, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.2819610581130005, + "p10_recall_at_k": 0.9, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.9876160990712073, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9938080495356038, + "vectors": 6260 + }, + { + "bytes_read_per_query": 308866.1300309597, + "depth": 21, + "doc_recall_at_k": 0.13547504873951974, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 21.393188854489164, + "metric": "cosine", + "ndcg_at_k": 0.28254183657573295, + "p10_recall_at_k": 0.7, + "p90_depth": 31, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 323, + "recall_at_k": 0.8894736842105271, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9907120743034056, + "vectors": 6260 + }, + { + "bytes_read_per_query": 319343.33126934984, + "depth": 48, + "doc_recall_at_k": 0.13628707671445503, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 48.12074303405573, + "metric": "cosine", + "ndcg_at_k": 0.28141257009743464, + "p10_recall_at_k": 0.9, + "p90_depth": 81, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 323, + "recall_at_k": 0.9708978328173375, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9938080495356038, + "vectors": 6260 + }, + { + "bytes_read_per_query": 341641.213622291, + "depth": 105, + "doc_recall_at_k": 0.13664357191255458, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 105.0030959752322, + "metric": "cosine", + "ndcg_at_k": 0.2820740712640988, + "p10_recall_at_k": 1.0, + "p90_depth": 204, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 323, + "recall_at_k": 0.9928792569659443, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9938080495356038, + "vectors": 6260 + }, + { + "bytes_read_per_query": 331200.0, + "depth": 20, + "doc_recall_at_k": 0.1350527163411749, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.28296073137923217, + "p10_recall_at_k": 0.7, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.8693498452012395, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9938080495356038, + "vectors": 6260 + }, + { + "bytes_read_per_query": 377280.0, + "depth": 50, + "doc_recall_at_k": 0.1362283782408485, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.28134394718218203, + "p10_recall_at_k": 0.9, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.9650154798761613, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9969040247678018, + "vectors": 6260 + }, + { + "bytes_read_per_query": 454080.0, + "depth": 100, + "doc_recall_at_k": 0.13659637276058328, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.28184911293791765, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.9910216718266252, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 6260 + }, + { + "bytes_read_per_query": 333339.93808049534, + "depth": 21, + "doc_recall_at_k": 0.13483222428059738, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 21.393188854489164, + "metric": "cosine", + "ndcg_at_k": 0.282016671534668, + "p10_recall_at_k": 0.7, + "p90_depth": 31, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 323, + "recall_at_k": 0.889783281733747, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9969040247678018, + "vectors": 6260 + }, + { + "bytes_read_per_query": 374393.46130030963, + "depth": 48, + "doc_recall_at_k": 0.1363793957241327, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 48.12074303405573, + "metric": "cosine", + "ndcg_at_k": 0.28129716009754296, + "p10_recall_at_k": 0.9, + "p90_depth": 81, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 323, + "recall_at_k": 0.9736842105263158, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 6260 + }, + { + "bytes_read_per_query": 461764.7554179566, + "depth": 105, + "doc_recall_at_k": 0.1366825120389185, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 105.0030959752322, + "metric": "cosine", + "ndcg_at_k": 0.2817570046541726, + "p10_recall_at_k": 1.0, + "p90_depth": 204, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 323, + "recall_at_k": 0.9962848297213621, + "rescore_ms": 1, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 6260 + }, + { + "bytes_read_per_query": 300480.0, + "depth": 0, + "doc_recall_at_k": 0.13757751316347785, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.2786296514908264, + "p10_recall_at_k": 0.6, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 323, + "recall_at_k": 0.7424148606811146, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.7275541795665634, + "vectors": 6260 + }, + { + "bytes_read_per_query": 308320.0, + "depth": 20, + "doc_recall_at_k": 0.13763616798998027, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.28378486501937955, + "p10_recall_at_k": 0.8, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.9216718266253878, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9845201238390093, + "vectors": 6260 + }, + { + "bytes_read_per_query": 320080.0, + "depth": 50, + "doc_recall_at_k": 0.13729352435999384, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.28274520343141396, + "p10_recall_at_k": 1.0, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.9907120743034054, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9938080495356038, + "vectors": 6260 + }, + { + "bytes_read_per_query": 339680.0, + "depth": 100, + "doc_recall_at_k": 0.13676741092184252, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.2823117419511956, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.9953560371517027, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9938080495356038, + "vectors": 6260 + }, + { + "bytes_read_per_query": 307301.7708978328, + "depth": 17, + "doc_recall_at_k": 0.13634630059011132, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 17.402476780185758, + "metric": "cosine", + "ndcg_at_k": 0.28170094933181833, + "p10_recall_at_k": 0.8, + "p90_depth": 23, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 323, + "recall_at_k": 0.902476780185759, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9876160990712074, + "vectors": 6260 + }, + { + "bytes_read_per_query": 312532.48297213623, + "depth": 31, + "doc_recall_at_k": 0.1368508967887378, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 30.74613003095975, + "metric": "cosine", + "ndcg_at_k": 0.2827649016475982, + "p10_recall_at_k": 0.9, + "p90_depth": 47, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 323, + "recall_at_k": 0.9777089783281737, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9907120743034056, + "vectors": 6260 + }, + { + "bytes_read_per_query": 321451.3931888545, + "depth": 53, + "doc_recall_at_k": 0.13545887237054308, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 53.4984520123839, + "metric": "cosine", + "ndcg_at_k": 0.28216037894380125, + "p10_recall_at_k": 1.0, + "p90_depth": 92, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 323, + "recall_at_k": 0.9941176470588236, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9938080495356038, + "vectors": 6260 + }, + { + "bytes_read_per_query": 331200.0, + "depth": 20, + "doc_recall_at_k": 0.13768265927544712, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.2836705802106119, + "p10_recall_at_k": 0.8, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.922910216718267, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9907120743034056, + "vectors": 6260 + }, + { + "bytes_read_per_query": 377280.0, + "depth": 50, + "doc_recall_at_k": 0.1388804521024568, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.2829541519199043, + "p10_recall_at_k": 1.0, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.9934984520123836, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 6260 + }, + { + "bytes_read_per_query": 454080.0, + "depth": 100, + "doc_recall_at_k": 0.13835433866430552, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.28252069043968603, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.9990712074303407, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 6260 + }, + { + "bytes_read_per_query": 327210.2043343653, + "depth": 17, + "doc_recall_at_k": 0.1364461707588919, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 17.402476780185758, + "metric": "cosine", + "ndcg_at_k": 0.28178832113308516, + "p10_recall_at_k": 0.8, + "p90_depth": 23, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 323, + "recall_at_k": 0.9030959752321986, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9938080495356038, + "vectors": 6260 + }, + { + "bytes_read_per_query": 347706.0557275542, + "depth": 31, + "doc_recall_at_k": 0.1384378245312008, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 30.74613003095975, + "metric": "cosine", + "ndcg_at_k": 0.28297385013608856, + "p10_recall_at_k": 0.9, + "p90_depth": 47, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 323, + "recall_at_k": 0.9798761609907124, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9969040247678018, + "vectors": 6260 + }, + { + "bytes_read_per_query": 382653.6222910217, + "depth": 53, + "doc_recall_at_k": 0.13704580011300604, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 53.4984520123839, + "metric": "cosine", + "ndcg_at_k": 0.2823693274322916, + "p10_recall_at_k": 1.0, + "p90_depth": 92, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 323, + "recall_at_k": 0.9978328173374613, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 6260 + }, + { + "bytes_read_per_query": 300480.0, + "depth": 0, + "doc_recall_at_k": 0.13380937648362437, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.27754611460054435, + "p10_recall_at_k": 0.5, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 323, + "recall_at_k": 0.6922600619195045, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.7120743034055728, + "vectors": 6260 + }, + { + "bytes_read_per_query": 308320.0, + "depth": 20, + "doc_recall_at_k": 0.13443353450103587, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.2821226187581341, + "p10_recall_at_k": 0.7, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.8693498452012393, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9876160990712074, + "vectors": 6260 + }, + { + "bytes_read_per_query": 320080.0, + "depth": 50, + "doc_recall_at_k": 0.1365186299076807, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.28161913330014443, + "p10_recall_at_k": 0.9, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.9637770897832821, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9907120743034056, + "vectors": 6260 + }, + { + "bytes_read_per_query": 339680.0, + "depth": 100, + "doc_recall_at_k": 0.1367781795313458, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.282317343654884, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.9876160990712074, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9938080495356038, + "vectors": 6260 + }, + { + "bytes_read_per_query": 308823.653250774, + "depth": 21, + "doc_recall_at_k": 0.13695805496742894, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 21.28482972136223, + "metric": "cosine", + "ndcg_at_k": 0.2834520346170087, + "p10_recall_at_k": 0.7, + "p90_depth": 31, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 323, + "recall_at_k": 0.8851393188854499, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9907120743034056, + "vectors": 6260 + }, + { + "bytes_read_per_query": 319140.6563467492, + "depth": 48, + "doc_recall_at_k": 0.13651594305344966, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 47.60371517027864, + "metric": "cosine", + "ndcg_at_k": 0.2821346289560808, + "p10_recall_at_k": 0.9, + "p90_depth": 81, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 323, + "recall_at_k": 0.9705882352941179, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9907120743034056, + "vectors": 6260 + }, + { + "bytes_read_per_query": 341089.0154798762, + "depth": 104, + "doc_recall_at_k": 0.13509558429645552, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 103.59442724458205, + "metric": "cosine", + "ndcg_at_k": 0.28154092903529154, + "p10_recall_at_k": 1.0, + "p90_depth": 203, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 323, + "recall_at_k": 0.9931888544891643, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9938080495356038, + "vectors": 6260 + }, + { + "bytes_read_per_query": 331200.0, + "depth": 20, + "doc_recall_at_k": 0.13418940742179444, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.28200865955763926, + "p10_recall_at_k": 0.7, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.8699690402476787, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9938080495356038, + "vectors": 6260 + }, + { + "bytes_read_per_query": 377280.0, + "depth": 50, + "doc_recall_at_k": 0.1366258265898573, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.28172443094834476, + "p10_recall_at_k": 0.9, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.9659442724458208, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9969040247678018, + "vectors": 6260 + }, + { + "bytes_read_per_query": 454080.0, + "depth": 100, + "doc_recall_at_k": 0.1368778250544195, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.28220539847980114, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.9910216718266253, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 6260 + }, + { + "bytes_read_per_query": 333173.49845201237, + "depth": 21, + "doc_recall_at_k": 0.1365641226350166, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 21.28482972136223, + "metric": "cosine", + "ndcg_at_k": 0.28291871652561723, + "p10_recall_at_k": 0.7, + "p90_depth": 31, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 323, + "recall_at_k": 0.8854489164086697, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9969040247678018, + "vectors": 6260 + }, + { + "bytes_read_per_query": 373599.306501548, + "depth": 48, + "doc_recall_at_k": 0.1366689674598371, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 47.60371517027864, + "metric": "cosine", + "ndcg_at_k": 0.2822399266042811, + "p10_recall_at_k": 0.9, + "p90_depth": 81, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 323, + "recall_at_k": 0.9730650154798762, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9969040247678018, + "vectors": 6260 + }, + { + "bytes_read_per_query": 459601.04024767806, + "depth": 104, + "doc_recall_at_k": 0.1366825120389185, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 103.59442724458205, + "metric": "cosine", + "ndcg_at_k": 0.2817498775237819, + "p10_recall_at_k": 1.0, + "p90_depth": 203, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 323, + "recall_at_k": 0.9969040247678018, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 6260 + }, + { + "bytes_read_per_query": 300480.0, + "depth": 0, + "doc_recall_at_k": 0.12771392558270395, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.27385814654279567, + "p10_recall_at_k": 0.6, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 323, + "recall_at_k": 0.7609907120743044, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.7151702786377709, + "vectors": 6260 + }, + { + "bytes_read_per_query": 308320.0, + "depth": 20, + "doc_recall_at_k": 0.13765356639660029, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.2827591162766326, + "p10_recall_at_k": 0.8, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.9427244582043338, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9938080495356038, + "vectors": 6260 + }, + { + "bytes_read_per_query": 320080.0, + "depth": 50, + "doc_recall_at_k": 0.13556342055376547, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.28195065125193475, + "p10_recall_at_k": 1.0, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.9922600619195048, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9938080495356038, + "vectors": 6260 + }, + { + "bytes_read_per_query": 339680.0, + "depth": 100, + "doc_recall_at_k": 0.13556342055376547, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.28195065125193475, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.9956656346749226, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9938080495356038, + "vectors": 6260 + }, + { + "bytes_read_per_query": 306559.03405572753, + "depth": 16, + "doc_recall_at_k": 0.1360659476353692, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 15.507739938080496, + "metric": "cosine", + "ndcg_at_k": 0.28103888382236586, + "p10_recall_at_k": 0.8, + "p90_depth": 20, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 323, + "recall_at_k": 0.9024767801857588, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9876160990712074, + "vectors": 6260 + }, + { + "bytes_read_per_query": 309987.51702786377, + "depth": 24, + "doc_recall_at_k": 0.13671002384397493, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 24.25386996904025, + "metric": "cosine", + "ndcg_at_k": 0.2822575862504232, + "p10_recall_at_k": 0.9, + "p90_depth": 35, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 323, + "recall_at_k": 0.9693498452012381, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9938080495356038, + "vectors": 6260 + }, + { + "bytes_read_per_query": 315386.9226006192, + "depth": 38, + "doc_recall_at_k": 0.1353030983120191, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 38.027863777089784, + "metric": "cosine", + "ndcg_at_k": 0.2819793055372944, + "p10_recall_at_k": 0.9, + "p90_depth": 61, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 323, + "recall_at_k": 0.9885448916408671, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9938080495356038, + "vectors": 6260 + }, + { + "bytes_read_per_query": 331200.0, + "depth": 20, + "doc_recall_at_k": 0.13775343656538086, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.28282439298411216, + "p10_recall_at_k": 0.8, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.9442724458204329, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 6260 + }, + { + "bytes_read_per_query": 377280.0, + "depth": 50, + "doc_recall_at_k": 0.13715034829622844, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.2821595997404251, + "p10_recall_at_k": 1.0, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.9959752321981425, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 6260 + }, + { + "bytes_read_per_query": 454080.0, + "depth": 100, + "doc_recall_at_k": 0.13715034829622844, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.2821595997404251, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.9993808049535604, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 6260 + }, + { + "bytes_read_per_query": 324299.8885448916, + "depth": 16, + "doc_recall_at_k": 0.13587201607293095, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 15.507739938080496, + "metric": "cosine", + "ndcg_at_k": 0.28114161537437493, + "p10_recall_at_k": 0.8, + "p90_depth": 20, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 323, + "recall_at_k": 0.9030959752321984, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9938080495356038, + "vectors": 6260 + }, + { + "bytes_read_per_query": 337733.9442724458, + "depth": 24, + "doc_recall_at_k": 0.13835033046975168, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 24.25386996904025, + "metric": "cosine", + "ndcg_at_k": 0.2826635031234304, + "p10_recall_at_k": 0.9, + "p90_depth": 35, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 323, + "recall_at_k": 0.9721362229102164, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 6260 + }, + { + "bytes_read_per_query": 358890.79876160994, + "depth": 38, + "doc_recall_at_k": 0.13843801367058114, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 38.027863777089784, + "metric": "cosine", + "ndcg_at_k": 0.2827369824678406, + "p10_recall_at_k": 1.0, + "p90_depth": 61, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 323, + "recall_at_k": 0.9922600619195047, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 6260 + }, + { + "bytes_read_per_query": 300480.0, + "depth": 0, + "doc_recall_at_k": 0.1283759548917701, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.2639937955894636, + "p10_recall_at_k": 0.6, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 323, + "recall_at_k": 0.779566563467493, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.6904024767801857, + "vectors": 6260 + }, + { + "bytes_read_per_query": 308320.0, + "depth": 20, + "doc_recall_at_k": 0.13597649457121613, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.282935399766218, + "p10_recall_at_k": 0.8, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.9414860681114557, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9938080495356038, + "vectors": 6260 + }, + { + "bytes_read_per_query": 320080.0, + "depth": 50, + "doc_recall_at_k": 0.13556342055376547, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.28199268823921186, + "p10_recall_at_k": 1.0, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.9894736842105264, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9938080495356038, + "vectors": 6260 + }, + { + "bytes_read_per_query": 339680.0, + "depth": 100, + "doc_recall_at_k": 0.13521942330574344, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.28174030016550045, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.9956656346749226, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9938080495356038, + "vectors": 6260 + }, + { + "bytes_read_per_query": 306226.50154798763, + "depth": 15, + "doc_recall_at_k": 0.13536433639786255, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 14.659442724458204, + "metric": "cosine", + "ndcg_at_k": 0.2818491532692474, + "p10_recall_at_k": 0.7, + "p90_depth": 19, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 323, + "recall_at_k": 0.8978328173374627, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9938080495356038, + "vectors": 6260 + }, + { + "bytes_read_per_query": 309009.3374613003, + "depth": 22, + "doc_recall_at_k": 0.13537879808017309, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 21.758513931888544, + "metric": "cosine", + "ndcg_at_k": 0.28200286266376756, + "p10_recall_at_k": 0.9, + "p90_depth": 31, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 323, + "recall_at_k": 0.9603715170278643, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9938080495356038, + "vectors": 6260 + }, + { + "bytes_read_per_query": 312982.7368421053, + "depth": 32, + "doc_recall_at_k": 0.13521942330574344, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 31.894736842105264, + "metric": "cosine", + "ndcg_at_k": 0.2818609053497159, + "p10_recall_at_k": 0.9, + "p90_depth": 49, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 323, + "recall_at_k": 0.982043343653251, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9938080495356038, + "vectors": 6260 + }, + { + "bytes_read_per_query": 331200.0, + "depth": 20, + "doc_recall_at_k": 0.1360763647399967, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.28301626268694624, + "p10_recall_at_k": 0.8, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.9430340557275548, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 6260 + }, + { + "bytes_read_per_query": 377280.0, + "depth": 50, + "doc_recall_at_k": 0.13715034829622844, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.2822016367277022, + "p10_recall_at_k": 1.0, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.9931888544891642, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 6260 + }, + { + "bytes_read_per_query": 454080.0, + "depth": 100, + "doc_recall_at_k": 0.13680635104820643, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.28194924865399074, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.9993808049535604, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 6260 + }, + { + "bytes_read_per_query": 322996.9040247678, + "depth": 15, + "doc_recall_at_k": 0.13546420656664312, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 14.659442724458204, + "metric": "cosine", + "ndcg_at_k": 0.28193001618997565, + "p10_recall_at_k": 0.7, + "p90_depth": 19, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 323, + "recall_at_k": 0.8990712074303419, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 6260 + }, + { + "bytes_read_per_query": 333901.0773993808, + "depth": 22, + "doc_recall_at_k": 0.13696572582263608, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 21.758513931888544, + "metric": "cosine", + "ndcg_at_k": 0.28221181115225796, + "p10_recall_at_k": 0.9, + "p90_depth": 31, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 323, + "recall_at_k": 0.9631578947368425, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 6260 + }, + { + "bytes_read_per_query": 349470.3157894737, + "depth": 32, + "doc_recall_at_k": 0.13680635104820643, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 323, + "mean_depth": 31.894736842105264, + "metric": "cosine", + "ndcg_at_k": 0.28206985383820626, + "p10_recall_at_k": 0.9, + "p90_depth": 49, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 323, + "recall_at_k": 0.9854489164086687, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 6260 + }, + { + "bytes_read_per_query": 150240.0, + "depth": 0, + "doc_recall_at_k": 0.12050051484895005, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 323, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.2544200174464856, + "p10_recall_at_k": 0.4, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 323, + "recall_at_k": 0.5944272445820431, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.5170278637770898, + "vectors": 6260 + }, + { + "bytes_read_per_query": 158080.0, + "depth": 20, + "doc_recall_at_k": 0.13158968993377898, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 323, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.2812403179711474, + "p10_recall_at_k": 0.6, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.7891640866873063, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9628482972136223, + "vectors": 6260 + }, + { + "bytes_read_per_query": 169840.0, + "depth": 50, + "doc_recall_at_k": 0.13419548878219, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 323, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.2819054417018927, + "p10_recall_at_k": 0.8, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.9359133126934994, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.978328173374613, + "vectors": 6260 + }, + { + "bytes_read_per_query": 189440.0, + "depth": 100, + "doc_recall_at_k": 0.13376127848737246, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 323, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.28142400319823596, + "p10_recall_at_k": 0.9, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.980185758513932, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9876160990712074, + "vectors": 6260 + }, + { + "bytes_read_per_query": 161283.9628482972, + "depth": 28, + "doc_recall_at_k": 0.13192590312459315, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 323, + "mean_depth": 28.173374613003094, + "metric": "cosine", + "ndcg_at_k": 0.2823785221783484, + "p10_recall_at_k": 0.7, + "p90_depth": 45, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 323, + "recall_at_k": 0.8588235294117653, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9721362229102167, + "vectors": 6260 + }, + { + "bytes_read_per_query": 180434.9226006192, + "depth": 77, + "doc_recall_at_k": 0.1340138117408417, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 323, + "mean_depth": 77.02786377708978, + "metric": "cosine", + "ndcg_at_k": 0.2813910915962346, + "p10_recall_at_k": 0.9, + "p90_depth": 149, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 323, + "recall_at_k": 0.9678018575851394, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9876160990712074, + "vectors": 6260 + }, + { + "bytes_read_per_query": 227539.24458204335, + "depth": 197, + "doc_recall_at_k": 0.13511381011556123, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 323, + "mean_depth": 197.19195046439629, + "metric": "cosine", + "ndcg_at_k": 0.28170724273879594, + "p10_recall_at_k": 1.0, + "p90_depth": 421, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 323, + "recall_at_k": 0.9925696594427246, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9938080495356038, + "vectors": 6260 + }, + { + "bytes_read_per_query": 180960.0, + "depth": 20, + "doc_recall_at_k": 0.13168956010255958, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 323, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.28138691369431285, + "p10_recall_at_k": 0.6, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.7894736842105261, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9690402476780186, + "vectors": 6260 + }, + { + "bytes_read_per_query": 227040.0, + "depth": 50, + "doc_recall_at_k": 0.13424198006765684, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 323, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.28182097766599196, + "p10_recall_at_k": 0.8, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.9374613003095984, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9845201238390093, + "vectors": 6260 + }, + { + "bytes_read_per_query": 303840.0, + "depth": 100, + "doc_recall_at_k": 0.13534820622983543, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 323, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.28165175995522984, + "p10_recall_at_k": 0.9, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.9832817337461301, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9938080495356038, + "vectors": 6260 + }, + { + "bytes_read_per_query": 193514.30340557275, + "depth": 28, + "doc_recall_at_k": 0.1318725242412794, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 323, + "mean_depth": 28.173374613003094, + "metric": "cosine", + "ndcg_at_k": 0.28206799629395596, + "p10_recall_at_k": 0.7, + "p90_depth": 45, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 323, + "recall_at_k": 0.8591331269349852, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.978328173374613, + "vectors": 6260 + }, + { + "bytes_read_per_query": 268554.7987616099, + "depth": 77, + "doc_recall_at_k": 0.13560829064240762, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 323, + "mean_depth": 77.02786377708978, + "metric": "cosine", + "ndcg_at_k": 0.2818010012308625, + "p10_recall_at_k": 0.9, + "p90_depth": 149, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 323, + "recall_at_k": 0.9696594427244585, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9938080495356038, + "vectors": 6260 + }, + { + "bytes_read_per_query": 453126.8359133127, + "depth": 197, + "doc_recall_at_k": 0.13670073785802422, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 323, + "mean_depth": 197.19195046439629, + "metric": "cosine", + "ndcg_at_k": 0.28191619122728634, + "p10_recall_at_k": 1.0, + "p90_depth": 421, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 323, + "recall_at_k": 0.9959752321981425, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 6260 + }, + { + "bytes_read_per_query": 150240.0, + "depth": 0, + "doc_recall_at_k": 0.11749525432094751, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 323, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.24740131228319043, + "p10_recall_at_k": 0.4, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 323, + "recall_at_k": 0.6365325077399383, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.5386996904024768, + "vectors": 6260 + }, + { + "bytes_read_per_query": 158080.0, + "depth": 20, + "doc_recall_at_k": 0.12576179849749455, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 323, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.27511536515755475, + "p10_recall_at_k": 0.6, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.8111455108359142, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9535603715170279, + "vectors": 6260 + }, + { + "bytes_read_per_query": 169840.0, + "depth": 50, + "doc_recall_at_k": 0.1361076821201302, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 323, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.2822685182919894, + "p10_recall_at_k": 0.8, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.9433436532507745, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9845201238390093, + "vectors": 6260 + }, + { + "bytes_read_per_query": 189440.0, + "depth": 100, + "doc_recall_at_k": 0.1353040995514104, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 323, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.28154936002110287, + "p10_recall_at_k": 0.9, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.9798761609907121, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9907120743034056, + "vectors": 6260 + }, + { + "bytes_read_per_query": 159531.49226006193, + "depth": 24, + "doc_recall_at_k": 0.12806375660321767, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 323, + "mean_depth": 23.70278637770898, + "metric": "cosine", + "ndcg_at_k": 0.27635285135413556, + "p10_recall_at_k": 0.7, + "p90_depth": 34, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 323, + "recall_at_k": 0.8523219814241497, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9721362229102167, + "vectors": 6260 + }, + { + "bytes_read_per_query": 171041.48606811147, + "depth": 53, + "doc_recall_at_k": 0.13382624711989993, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 323, + "mean_depth": 53.06501547987616, + "metric": "cosine", + "ndcg_at_k": 0.28171640342962, + "p10_recall_at_k": 0.9, + "p90_depth": 88, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 323, + "recall_at_k": 0.954179566563468, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9907120743034056, + "vectors": 6260 + }, + { + "bytes_read_per_query": 195633.1145510836, + "depth": 116, + "doc_recall_at_k": 0.1349098257825236, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 323, + "mean_depth": 115.79876160990712, + "metric": "cosine", + "ndcg_at_k": 0.2817878908392067, + "p10_recall_at_k": 0.9, + "p90_depth": 222, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 323, + "recall_at_k": 0.986996904024768, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9938080495356038, + "vectors": 6260 + }, + { + "bytes_read_per_query": 180960.0, + "depth": 20, + "doc_recall_at_k": 0.12586166866627516, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 323, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.2751943349121622, + "p10_recall_at_k": 0.6, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.8114551083591343, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9597523219814241, + "vectors": 6260 + }, + { + "bytes_read_per_query": 227040.0, + "depth": 50, + "doc_recall_at_k": 0.13775553990500986, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 323, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.28286964600649966, + "p10_recall_at_k": 0.8, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.9452012383900933, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9907120743034056, + "vectors": 6260 + }, + { + "bytes_read_per_query": 303840.0, + "depth": 100, + "doc_recall_at_k": 0.1384923937932862, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 323, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.28250400533616593, + "p10_recall_at_k": 0.9, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.9829721362229102, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9969040247678018, + "vectors": 6260 + }, + { + "bytes_read_per_query": 186647.479876161, + "depth": 24, + "doc_recall_at_k": 0.12972238470041683, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 323, + "mean_depth": 23.70278637770898, + "metric": "cosine", + "ndcg_at_k": 0.27691985060467667, + "p10_recall_at_k": 0.7, + "p90_depth": 34, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 323, + "recall_at_k": 0.8529411764705893, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.978328173374613, + "vectors": 6260 + }, + { + "bytes_read_per_query": 231747.86377708978, + "depth": 53, + "doc_recall_at_k": 0.13702209252087866, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 323, + "mean_depth": 53.06501547987616, + "metric": "cosine", + "ndcg_at_k": 0.28285642367757197, + "p10_recall_at_k": 0.9, + "p90_depth": 88, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 323, + "recall_at_k": 0.9563467492260066, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9969040247678018, + "vectors": 6260 + }, + { + "bytes_read_per_query": 328106.89783281734, + "depth": 116, + "doc_recall_at_k": 0.13650430468408953, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 323, + "mean_depth": 115.79876160990712, + "metric": "cosine", + "ndcg_at_k": 0.28219380771221386, + "p10_recall_at_k": 1.0, + "p90_depth": 222, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 323, + "recall_at_k": 0.9900928792569662, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 6260 + }, + { + "bytes_read_per_query": 150240.0, + "depth": 0, + "doc_recall_at_k": 0.1085930693714295, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 323, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.23762630403274157, + "p10_recall_at_k": 0.4, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 323, + "recall_at_k": 0.6210526315789473, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.5510835913312694, + "vectors": 6260 + }, + { + "bytes_read_per_query": 158080.0, + "depth": 20, + "doc_recall_at_k": 0.12653888476029376, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 323, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.2731943793767632, + "p10_recall_at_k": 0.6, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.8201238390092891, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9628482972136223, + "vectors": 6260 + }, + { + "bytes_read_per_query": 169840.0, + "depth": 50, + "doc_recall_at_k": 0.13294520804492718, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 323, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.2802163478091858, + "p10_recall_at_k": 0.8, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.9445820433436534, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9845201238390093, + "vectors": 6260 + }, + { + "bytes_read_per_query": 189440.0, + "depth": 100, + "doc_recall_at_k": 0.13450080927443592, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 323, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.2809016746959515, + "p10_recall_at_k": 0.9, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.9755417956656347, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9876160990712074, + "vectors": 6260 + }, + { + "bytes_read_per_query": 158741.4241486068, + "depth": 22, + "doc_recall_at_k": 0.12375718999778869, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 323, + "mean_depth": 21.687306501547987, + "metric": "cosine", + "ndcg_at_k": 0.27065120589617275, + "p10_recall_at_k": 0.7, + "p90_depth": 31, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 323, + "recall_at_k": 0.8402476780185768, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9628482972136223, + "vectors": 6260 + }, + { + "bytes_read_per_query": 168329.04024767803, + "depth": 46, + "doc_recall_at_k": 0.1329333847908189, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 323, + "mean_depth": 46.14551083591331, + "metric": "cosine", + "ndcg_at_k": 0.2803148259380811, + "p10_recall_at_k": 0.8, + "p90_depth": 74, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 323, + "recall_at_k": 0.9427244582043346, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9814241486068112, + "vectors": 6260 + }, + { + "bytes_read_per_query": 187880.49535603717, + "depth": 96, + "doc_recall_at_k": 0.13445053275357116, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 323, + "mean_depth": 96.0216718266254, + "metric": "cosine", + "ndcg_at_k": 0.2812231839272726, + "p10_recall_at_k": 0.9, + "p90_depth": 175, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 323, + "recall_at_k": 0.9783281733746133, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9907120743034056, + "vectors": 6260 + }, + { + "bytes_read_per_query": 180960.0, + "depth": 20, + "doc_recall_at_k": 0.12658881984468406, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 323, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.2730883865543036, + "p10_recall_at_k": 0.6, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.8207430340557287, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9690402476780186, + "vectors": 6260 + }, + { + "bytes_read_per_query": 227040.0, + "depth": 50, + "doc_recall_at_k": 0.13459306582980682, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 323, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.28083890488393837, + "p10_recall_at_k": 0.8, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.9470588235294118, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9907120743034056, + "vectors": 6260 + }, + { + "bytes_read_per_query": 303840.0, + "depth": 100, + "doc_recall_at_k": 0.13608773701689889, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 323, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.28112045909305605, + "p10_recall_at_k": 0.9, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.9789473684210528, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9938080495356038, + "vectors": 6260 + }, + { + "bytes_read_per_query": 183551.7027863777, + "depth": 22, + "doc_recall_at_k": 0.12376031094056308, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 323, + "mean_depth": 21.687306501547987, + "metric": "cosine", + "ndcg_at_k": 0.2705488160133385, + "p10_recall_at_k": 0.7, + "p90_depth": 31, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 323, + "recall_at_k": 0.8405572755417966, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9690402476780186, + "vectors": 6260 + }, + { + "bytes_read_per_query": 221119.50464396283, + "depth": 46, + "doc_recall_at_k": 0.13452786369238479, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 323, + "mean_depth": 46.14551083591331, + "metric": "cosine", + "ndcg_at_k": 0.2807305787197025, + "p10_recall_at_k": 0.8, + "p90_depth": 74, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 323, + "recall_at_k": 0.9448916408668732, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9876160990712074, + "vectors": 6260 + }, + { + "bytes_read_per_query": 297729.2879256966, + "depth": 96, + "doc_recall_at_k": 0.1360374604960341, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 323, + "mean_depth": 96.0216718266254, + "metric": "cosine", + "ndcg_at_k": 0.28144196832437707, + "p10_recall_at_k": 0.9, + "p90_depth": 175, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 323, + "recall_at_k": 0.9817337461300312, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9969040247678018, + "vectors": 6260 + }, + { + "bytes_read_per_query": 600960.0, + "depth": 0, + "doc_recall_at_k": 0.13579010339617162, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 323, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.2834225478039319, + "p10_recall_at_k": 0.8, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 323, + "recall_at_k": 0.8743034055727568, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.9009287925696594, + "vectors": 6260 + }, + { + "bytes_read_per_query": 608800.0, + "depth": 20, + "doc_recall_at_k": 0.13676741092184252, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 323, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.2822890286075563, + "p10_recall_at_k": 0.9, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.9885448916408671, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9938080495356038, + "vectors": 6260 + }, + { + "bytes_read_per_query": 620560.0, + "depth": 50, + "doc_recall_at_k": 0.13521942330574344, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 323, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.28174030016550045, + "p10_recall_at_k": 1.0, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.9962848297213622, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9938080495356038, + "vectors": 6260 + }, + { + "bytes_read_per_query": 640160.0, + "depth": 100, + "doc_recall_at_k": 0.13521942330574344, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 323, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.28174030016550045, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.9962848297213622, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9938080495356038, + "vectors": 6260 + }, + { + "bytes_read_per_query": 605990.4643962848, + "depth": 13, + "doc_recall_at_k": 0.1389452593362079, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 323, + "mean_depth": 12.8328173374613, + "metric": "cosine", + "ndcg_at_k": 0.28473547392824217, + "p10_recall_at_k": 0.9, + "p90_depth": 16, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 323, + "recall_at_k": 0.9501547987616106, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9938080495356038, + "vectors": 6260 + }, + { + "bytes_read_per_query": 607372.7801857585, + "depth": 16, + "doc_recall_at_k": 0.13718891330388117, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 323, + "mean_depth": 16.359133126934985, + "metric": "cosine", + "ndcg_at_k": 0.28268834653668584, + "p10_recall_at_k": 0.9, + "p90_depth": 22, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 323, + "recall_at_k": 0.9777089783281736, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9938080495356038, + "vectors": 6260 + }, + { + "bytes_read_per_query": 609127.6780185759, + "depth": 21, + "doc_recall_at_k": 0.1368885892222988, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 323, + "mean_depth": 20.8359133126935, + "metric": "cosine", + "ndcg_at_k": 0.2820560823051581, + "p10_recall_at_k": 1.0, + "p90_depth": 29, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 323, + "recall_at_k": 0.9913312693498454, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9938080495356038, + "vectors": 6260 + }, + { + "bytes_read_per_query": 631680.0, + "depth": 20, + "doc_recall_at_k": 0.13680635104820643, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 323, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.28197196199763014, + "p10_recall_at_k": 1.0, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.9913312693498454, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 6260 + }, + { + "bytes_read_per_query": 677760.0, + "depth": 50, + "doc_recall_at_k": 0.13680635104820643, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 323, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.28194924865399074, + "p10_recall_at_k": 1.0, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 1.0, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 6260 + }, + { + "bytes_read_per_query": 754560.0, + "depth": 100, + "doc_recall_at_k": 0.13680635104820643, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 323, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.28194924865399074, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 1.0, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 6260 + }, + { + "bytes_read_per_query": 620671.2074303406, + "depth": 13, + "doc_recall_at_k": 0.13898419946257184, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 323, + "mean_depth": 12.8328173374613, + "metric": "cosine", + "ndcg_at_k": 0.2844329314524478, + "p10_recall_at_k": 0.9, + "p90_depth": 16, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 323, + "recall_at_k": 0.9523219814241493, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 6260 + }, + { + "bytes_read_per_query": 626087.6284829722, + "depth": 16, + "doc_recall_at_k": 0.13717791834585477, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 323, + "mean_depth": 16.359133126934985, + "metric": "cosine", + "ndcg_at_k": 0.2821789997677604, + "p10_recall_at_k": 0.9, + "p90_depth": 22, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 323, + "recall_at_k": 0.980495356037152, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 6260 + }, + { + "bytes_read_per_query": 632963.9628482972, + "depth": 21, + "doc_recall_at_k": 0.13692752934866267, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 323, + "mean_depth": 20.8359133126935, + "metric": "cosine", + "ndcg_at_k": 0.28173901569523196, + "p10_recall_at_k": 1.0, + "p90_depth": 29, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 323, + "recall_at_k": 0.9944272445820435, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 6260 + }, + { + "bytes_read_per_query": 600960.0, + "depth": 0, + "doc_recall_at_k": 0.13882279311688525, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 323, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.2809643700253817, + "p10_recall_at_k": 0.7, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 323, + "recall_at_k": 0.883281733746131, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.8947368421052632, + "vectors": 6260 + }, + { + "bytes_read_per_query": 608800.0, + "depth": 20, + "doc_recall_at_k": 0.13556342055376547, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 323, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.2819686402108754, + "p10_recall_at_k": 1.0, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.992260061919505, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9938080495356038, + "vectors": 6260 + }, + { + "bytes_read_per_query": 620560.0, + "depth": 50, + "doc_recall_at_k": 0.13521942330574344, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 323, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.28174030016550045, + "p10_recall_at_k": 1.0, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.9962848297213622, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9938080495356038, + "vectors": 6260 + }, + { + "bytes_read_per_query": 640160.0, + "depth": 100, + "doc_recall_at_k": 0.13521942330574344, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 323, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.28174030016550045, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.9962848297213622, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9938080495356038, + "vectors": 6260 + }, + { + "bytes_read_per_query": 605529.2879256966, + "depth": 12, + "doc_recall_at_k": 0.13705339000980343, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 323, + "mean_depth": 11.656346749226007, + "metric": "cosine", + "ndcg_at_k": 0.2822362291757218, + "p10_recall_at_k": 0.9, + "p90_depth": 14, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 323, + "recall_at_k": 0.9467492260061919, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9938080495356038, + "vectors": 6260 + }, + { + "bytes_read_per_query": 606314.5015479876, + "depth": 14, + "doc_recall_at_k": 0.1356810372598906, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 323, + "mean_depth": 13.659442724458204, + "metric": "cosine", + "ndcg_at_k": 0.2819639275618505, + "p10_recall_at_k": 0.9, + "p90_depth": 17, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 323, + "recall_at_k": 0.9764705882352942, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9938080495356038, + "vectors": 6260 + }, + { + "bytes_read_per_query": 607234.4272445821, + "depth": 16, + "doc_recall_at_k": 0.13521942330574344, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 323, + "mean_depth": 16.006191950464395, + "metric": "cosine", + "ndcg_at_k": 0.28175828912444106, + "p10_recall_at_k": 0.9, + "p90_depth": 21, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 323, + "recall_at_k": 0.9876160990712074, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9938080495356038, + "vectors": 6260 + }, + { + "bytes_read_per_query": 631680.0, + "depth": 20, + "doc_recall_at_k": 0.13869833591232753, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 323, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.2827263171414216, + "p10_recall_at_k": 1.0, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 0.9959752321981425, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 6260 + }, + { + "bytes_read_per_query": 677760.0, + "depth": 50, + "doc_recall_at_k": 0.13680635104820643, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 323, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.28194924865399074, + "p10_recall_at_k": 1.0, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 1.0, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 6260 + }, + { + "bytes_read_per_query": 754560.0, + "depth": 100, + "doc_recall_at_k": 0.13680635104820643, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 323, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.28194924865399074, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 323, + "recall_at_k": 1.0, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 6260 + }, + { + "bytes_read_per_query": 618864.1486068112, + "depth": 12, + "doc_recall_at_k": 0.13865453186350934, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 323, + "mean_depth": 11.656346749226007, + "metric": "cosine", + "ndcg_at_k": 0.2826592629333656, + "p10_recall_at_k": 0.9, + "p90_depth": 14, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 323, + "recall_at_k": 0.948297213622291, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 6260 + }, + { + "bytes_read_per_query": 621940.9040247678, + "depth": 14, + "doc_recall_at_k": 0.13732134388566733, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 323, + "mean_depth": 13.659442724458204, + "metric": "cosine", + "ndcg_at_k": 0.2823771254108801, + "p10_recall_at_k": 0.9, + "p90_depth": 17, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 323, + "recall_at_k": 0.9789473684210526, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 6260 + }, + { + "bytes_read_per_query": 625545.5108359134, + "depth": 16, + "doc_recall_at_k": 0.1383543386643055, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 323, + "mean_depth": 16.006191950464395, + "metric": "cosine", + "ndcg_at_k": 0.2825159660549873, + "p10_recall_at_k": 1.0, + "p90_depth": 21, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 323, + "recall_at_k": 0.9910216718266255, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 6260 + } + ], + "profiles": [ + { + "depth_for_recall": [ + [ + 0.9, + 28 + ], + [ + 0.95, + 43 + ], + [ + 0.99, + 100 + ], + [ + 1.0, + 291 + ] + ], + "first": "binary", + "first_bytes": 48, + "k": 10, + "median_full_depth": 31.0, + "metric": "cosine", + "neighbour_band": 0.030829548835754395, + "queries": 323, + "rank_noise": 0.029061802535522818, + "recall_at_k": 0.6690402476780186, + "vectors": 6260 + }, + { + "depth_for_recall": [ + [ + 0.9, + 25 + ], + [ + 0.95, + 40 + ], + [ + 0.99, + 91 + ], + [ + 1.0, + 263 + ] + ], + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "median_full_depth": 27.0, + "metric": "cosine", + "neighbour_band": 0.030829548835754395, + "queries": 323, + "rank_noise": 0.026947140886363668, + "recall_at_k": 0.6907120743034053, + "vectors": 6260 + }, + { + "depth_for_recall": [ + [ + 0.9, + 18 + ], + [ + 0.95, + 25 + ], + [ + 0.99, + 46 + ], + [ + 1.0, + 152 + ] + ], + "first": "pq48", + "first_bytes": 48, + "k": 10, + "median_full_depth": 19.0, + "metric": "cosine", + "neighbour_band": 0.030829548835754395, + "queries": 323, + "rank_noise": 0.02263408737491555, + "recall_at_k": 0.7424148606811146, + "vectors": 6260 + }, + { + "depth_for_recall": [ + [ + 0.9, + 25 + ], + [ + 0.95, + 39 + ], + [ + 0.99, + 94 + ], + [ + 1.0, + 316 + ] + ], + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "median_full_depth": 27.0, + "metric": "cosine", + "neighbour_band": 0.030829548835754395, + "queries": 323, + "rank_noise": 0.026935068795784896, + "recall_at_k": 0.6922600619195045, + "vectors": 6260 + }, + { + "depth_for_recall": [ + [ + 0.9, + 16 + ], + [ + 0.95, + 22 + ], + [ + 0.99, + 40 + ], + [ + 1.0, + 237 + ] + ], + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "median_full_depth": 17.0, + "metric": "cosine", + "neighbour_band": 0.030829548835754395, + "queries": 323, + "rank_noise": 0.022015476028613395, + "recall_at_k": 0.7609907120743044, + "vectors": 6260 + }, + { + "depth_for_recall": [ + [ + 0.9, + 15 + ], + [ + 0.95, + 22 + ], + [ + 0.99, + 43 + ], + [ + 1.0, + 237 + ] + ], + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "median_full_depth": 16.0, + "metric": "cosine", + "neighbour_band": 0.030829548835754395, + "queries": 323, + "rank_noise": 0.021317732713607016, + "recall_at_k": 0.779566563467493, + "vectors": 6260 + }, + { + "depth_for_recall": [ + [ + 0.9, + 36 + ], + [ + 0.95, + 59 + ], + [ + 0.99, + 136 + ], + [ + 1.0, + 1283 + ] + ], + "first": "pq24", + "first_bytes": 24, + "k": 10, + "median_full_depth": 39.0, + "metric": "cosine", + "neighbour_band": 0.030829548835754395, + "queries": 323, + "rank_noise": 0.03515438785727546, + "recall_at_k": 0.5944272445820431, + "vectors": 6260 + }, + { + "depth_for_recall": [ + [ + 0.9, + 33 + ], + [ + 0.95, + 54 + ], + [ + 0.99, + 143 + ], + [ + 1.0, + 1504 + ] + ], + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "median_full_depth": 36.0, + "metric": "cosine", + "neighbour_band": 0.030829548835754395, + "queries": 323, + "rank_noise": 0.03449976187953163, + "recall_at_k": 0.6365325077399383, + "vectors": 6260 + }, + { + "depth_for_recall": [ + [ + 0.9, + 33 + ], + [ + 0.95, + 54 + ], + [ + 0.99, + 153 + ], + [ + 1.0, + 1064 + ] + ], + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "median_full_depth": 34.0, + "metric": "cosine", + "neighbour_band": 0.030829548835754395, + "queries": 323, + "rank_noise": 0.03491761344350988, + "recall_at_k": 0.6210526315789473, + "vectors": 6260 + }, + { + "depth_for_recall": [ + [ + 0.9, + 11 + ], + [ + 0.95, + 13 + ], + [ + 0.99, + 20 + ], + [ + 1.0, + 46 + ] + ], + "first": "pq96", + "first_bytes": 96, + "k": 10, + "median_full_depth": 12.0, + "metric": "cosine", + "neighbour_band": 0.030829548835754395, + "queries": 323, + "rank_noise": 0.011645806522831125, + "recall_at_k": 0.8743034055727568, + "vectors": 6260 + }, + { + "depth_for_recall": [ + [ + 0.9, + 11 + ], + [ + 0.95, + 12 + ], + [ + 0.99, + 17 + ], + [ + 1.0, + 44 + ] + ], + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "median_full_depth": 12.0, + "metric": "cosine", + "neighbour_band": 0.030829548835754395, + "queries": 323, + "rank_noise": 0.010180275614254608, + "recall_at_k": 0.883281733746131, + "vectors": 6260 + } + ] +} diff --git a/experiments/2026-11-hybrid-adaptive-matrix/results/scifact-bge-small-k10.json b/experiments/2026-11-hybrid-adaptive-matrix/results/scifact-bge-small-k10.json new file mode 100644 index 0000000..efc5809 --- /dev/null +++ b/experiments/2026-11-hybrid-adaptive-matrix/results/scifact-bge-small-k10.json @@ -0,0 +1,3900 @@ +{ + "cells": [ + { + "bytes_read_per_query": 385728.0, + "depth": 0, + "doc_recall_at_k": 0.7676111111111111, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.6536746843702901, + "p10_recall_at_k": 0.4, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 300, + "recall_at_k": 0.6109999999999995, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.7, + "vectors": 8036 + }, + { + "bytes_read_per_query": 393568.0, + "depth": 20, + "doc_recall_at_k": 0.8002222222222222, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.6908633912021365, + "p10_recall_at_k": 0.6, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.7776666666666668, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9866666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 405328.0, + "depth": 50, + "doc_recall_at_k": 0.8195555555555556, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.6982262011446152, + "p10_recall_at_k": 0.8, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.9143333333333347, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9966666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 424928.0, + "depth": 100, + "doc_recall_at_k": 0.8212222222222223, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.6989925221637916, + "p10_recall_at_k": 0.9, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.9613333333333336, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9966666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 395680.88, + "depth": 25, + "doc_recall_at_k": 0.8135555555555556, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 25.39, + "metric": "cosine", + "ndcg_at_k": 0.6957158295120849, + "p10_recall_at_k": 0.7, + "p90_depth": 38, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 300, + "recall_at_k": 0.8220000000000012, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9966666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 410395.25333333336, + "depth": 63, + "doc_recall_at_k": 0.8245555555555556, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 62.92666666666667, + "metric": "cosine", + "ndcg_at_k": 0.7001084103508829, + "p10_recall_at_k": 0.9, + "p90_depth": 108, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 300, + "recall_at_k": 0.945000000000001, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9966666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 443222.64, + "depth": 147, + "doc_recall_at_k": 0.8178888888888889, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 146.67, + "metric": "cosine", + "ndcg_at_k": 0.6978365964864137, + "p10_recall_at_k": 0.9, + "p90_depth": 275, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 300, + "recall_at_k": 0.9793333333333337, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9966666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 416448.0, + "depth": 20, + "doc_recall_at_k": 0.7968888888888889, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.6901997721917655, + "p10_recall_at_k": 0.6, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.7790000000000001, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.99, + "vectors": 8036 + }, + { + "bytes_read_per_query": 462528.0, + "depth": 50, + "doc_recall_at_k": 0.8195555555555556, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.6985660154531242, + "p10_recall_at_k": 0.8, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.9193333333333347, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 539328.0, + "depth": 100, + "doc_recall_at_k": 0.8212222222222223, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.6993323364723006, + "p10_recall_at_k": 0.9, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.9700000000000002, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 424727.04, + "depth": 25, + "doc_recall_at_k": 0.8135555555555556, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 25.39, + "metric": "cosine", + "ndcg_at_k": 0.6959960822987686, + "p10_recall_at_k": 0.7, + "p90_depth": 38, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 300, + "recall_at_k": 0.8236666666666677, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 482383.36, + "depth": 63, + "doc_recall_at_k": 0.8245555555555556, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 62.92666666666667, + "metric": "cosine", + "ndcg_at_k": 0.7004482246593918, + "p10_recall_at_k": 0.9, + "p90_depth": 108, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 300, + "recall_at_k": 0.9510000000000008, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 611013.12, + "depth": 147, + "doc_recall_at_k": 0.8178888888888889, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 146.67, + "metric": "cosine", + "ndcg_at_k": 0.698128294524517, + "p10_recall_at_k": 1.0, + "p90_depth": 275, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 300, + "recall_at_k": 0.9906666666666665, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 385728.0, + "depth": 0, + "doc_recall_at_k": 0.7695555555555555, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.6576686434556663, + "p10_recall_at_k": 0.4, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 300, + "recall_at_k": 0.6559999999999993, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.7266666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 393568.0, + "depth": 20, + "doc_recall_at_k": 0.8168888888888889, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.6965206494865323, + "p10_recall_at_k": 0.6, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.8296666666666687, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9833333333333333, + "vectors": 8036 + }, + { + "bytes_read_per_query": 405328.0, + "depth": 50, + "doc_recall_at_k": 0.8178888888888889, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.697876480384234, + "p10_recall_at_k": 0.9, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.9526666666666671, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9966666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 424928.0, + "depth": 100, + "doc_recall_at_k": 0.8178888888888889, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.697876480384234, + "p10_recall_at_k": 0.9, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.9806666666666672, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9966666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 396168.26666666666, + "depth": 27, + "doc_recall_at_k": 0.8218888888888889, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 26.633333333333333, + "metric": "cosine", + "ndcg_at_k": 0.6988798113214303, + "p10_recall_at_k": 0.8, + "p90_depth": 39, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 300, + "recall_at_k": 0.888000000000002, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9933333333333333, + "vectors": 8036 + }, + { + "bytes_read_per_query": 413042.56, + "depth": 70, + "doc_recall_at_k": 0.8178888888888889, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 69.68, + "metric": "cosine", + "ndcg_at_k": 0.697876480384234, + "p10_recall_at_k": 0.9, + "p90_depth": 120, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 300, + "recall_at_k": 0.976333333333334, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9966666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 453029.17333333334, + "depth": 172, + "doc_recall_at_k": 0.8178888888888889, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 171.68666666666667, + "metric": "cosine", + "ndcg_at_k": 0.6978365964864137, + "p10_recall_at_k": 0.9, + "p90_depth": 317, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 300, + "recall_at_k": 0.9863333333333336, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9966666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 416448.0, + "depth": 20, + "doc_recall_at_k": 0.8168888888888889, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.6967143846255784, + "p10_recall_at_k": 0.6, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.8310000000000021, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9866666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 462528.0, + "depth": 50, + "doc_recall_at_k": 0.8178888888888889, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.6981681784223372, + "p10_recall_at_k": 0.9, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.9610000000000002, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 539328.0, + "depth": 100, + "doc_recall_at_k": 0.8178888888888889, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.6981681784223372, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.9923333333333336, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 426636.8, + "depth": 27, + "doc_recall_at_k": 0.8185555555555556, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 26.633333333333333, + "metric": "cosine", + "ndcg_at_k": 0.6982560762088798, + "p10_recall_at_k": 0.8, + "p90_depth": 39, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 300, + "recall_at_k": 0.8920000000000018, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9966666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 492756.48, + "depth": 70, + "doc_recall_at_k": 0.8178888888888889, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 69.68, + "metric": "cosine", + "ndcg_at_k": 0.6981681784223372, + "p10_recall_at_k": 0.9, + "p90_depth": 120, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 300, + "recall_at_k": 0.9873333333333337, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 649438.72, + "depth": 172, + "doc_recall_at_k": 0.8178888888888889, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 171.68666666666667, + "metric": "cosine", + "ndcg_at_k": 0.698128294524517, + "p10_recall_at_k": 1.0, + "p90_depth": 317, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 300, + "recall_at_k": 0.9993333333333334, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 385728.0, + "depth": 0, + "doc_recall_at_k": 0.7968888888888889, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.6775694097419889, + "p10_recall_at_k": 0.5, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 300, + "recall_at_k": 0.711333333333333, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.8033333333333333, + "vectors": 8036 + }, + { + "bytes_read_per_query": 393568.0, + "depth": 20, + "doc_recall_at_k": 0.8045555555555556, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.6941995572398264, + "p10_recall_at_k": 0.7, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.8923333333333349, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9933333333333333, + "vectors": 8036 + }, + { + "bytes_read_per_query": 405328.0, + "depth": 50, + "doc_recall_at_k": 0.8145555555555556, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.6968134855435288, + "p10_recall_at_k": 0.9, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.9723333333333338, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9966666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 424928.0, + "depth": 100, + "doc_recall_at_k": 0.8178888888888889, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.697876480384234, + "p10_recall_at_k": 0.9, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.9853333333333336, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9966666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 393975.68, + "depth": 21, + "doc_recall_at_k": 0.8078888888888889, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 21.04, + "metric": "cosine", + "ndcg_at_k": 0.695222043314821, + "p10_recall_at_k": 0.8, + "p90_depth": 29, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 300, + "recall_at_k": 0.9073333333333348, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9966666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 402965.5466666667, + "depth": 44, + "doc_recall_at_k": 0.8178888888888889, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 43.973333333333336, + "metric": "cosine", + "ndcg_at_k": 0.6979245966546399, + "p10_recall_at_k": 0.9, + "p90_depth": 71, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 300, + "recall_at_k": 0.9733333333333339, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9966666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 420910.0, + "depth": 90, + "doc_recall_at_k": 0.8178888888888889, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 89.75, + "metric": "cosine", + "ndcg_at_k": 0.697876480384234, + "p10_recall_at_k": 0.9, + "p90_depth": 154, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 300, + "recall_at_k": 0.9840000000000003, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9966666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 416448.0, + "depth": 20, + "doc_recall_at_k": 0.8045555555555556, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.6945393715483354, + "p10_recall_at_k": 0.7, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.8963333333333348, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9966666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 462528.0, + "depth": 50, + "doc_recall_at_k": 0.8145555555555556, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.6971051835816321, + "p10_recall_at_k": 0.9, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.9836666666666667, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 539328.0, + "depth": 100, + "doc_recall_at_k": 0.8178888888888889, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.6981681784223372, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.9983333333333333, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 418045.44, + "depth": 21, + "doc_recall_at_k": 0.8078888888888889, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 21.04, + "metric": "cosine", + "ndcg_at_k": 0.69556185762333, + "p10_recall_at_k": 0.8, + "p90_depth": 29, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 300, + "recall_at_k": 0.912333333333335, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 453271.04000000004, + "depth": 44, + "doc_recall_at_k": 0.8178888888888889, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 43.973333333333336, + "metric": "cosine", + "ndcg_at_k": 0.6982162946927432, + "p10_recall_at_k": 0.9, + "p90_depth": 71, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 300, + "recall_at_k": 0.9853333333333332, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 523584.0, + "depth": 90, + "doc_recall_at_k": 0.8178888888888889, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 89.75, + "metric": "cosine", + "ndcg_at_k": 0.6981681784223372, + "p10_recall_at_k": 1.0, + "p90_depth": 154, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 300, + "recall_at_k": 0.9970000000000001, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 385728.0, + "depth": 0, + "doc_recall_at_k": 0.767888888888889, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.6579100547646629, + "p10_recall_at_k": 0.4, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 300, + "recall_at_k": 0.6533333333333328, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.73, + "vectors": 8036 + }, + { + "bytes_read_per_query": 393568.0, + "depth": 20, + "doc_recall_at_k": 0.8235555555555556, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.6976201089315751, + "p10_recall_at_k": 0.6, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.8273333333333354, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.98, + "vectors": 8036 + }, + { + "bytes_read_per_query": 405328.0, + "depth": 50, + "doc_recall_at_k": 0.8178888888888889, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.697876480384234, + "p10_recall_at_k": 0.9, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.9506666666666674, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9966666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 424928.0, + "depth": 100, + "doc_recall_at_k": 0.8178888888888889, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.697876480384234, + "p10_recall_at_k": 0.9, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.979666666666667, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9966666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 396156.50666666665, + "depth": 27, + "doc_recall_at_k": 0.8178888888888889, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 26.60333333333333, + "metric": "cosine", + "ndcg_at_k": 0.6981910543273419, + "p10_recall_at_k": 0.8, + "p90_depth": 39, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 300, + "recall_at_k": 0.8860000000000019, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9933333333333333, + "vectors": 8036 + }, + { + "bytes_read_per_query": 412983.76, + "depth": 70, + "doc_recall_at_k": 0.8178888888888889, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 69.53, + "metric": "cosine", + "ndcg_at_k": 0.697876480384234, + "p10_recall_at_k": 0.9, + "p90_depth": 119, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 300, + "recall_at_k": 0.9743333333333339, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9966666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 453076.2133333333, + "depth": 172, + "doc_recall_at_k": 0.8178888888888889, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 171.80666666666667, + "metric": "cosine", + "ndcg_at_k": 0.6978365964864137, + "p10_recall_at_k": 0.9, + "p90_depth": 312, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 300, + "recall_at_k": 0.9860000000000002, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9966666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 416448.0, + "depth": 20, + "doc_recall_at_k": 0.8202222222222222, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.6966192165098063, + "p10_recall_at_k": 0.6, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.8283333333333354, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9833333333333333, + "vectors": 8036 + }, + { + "bytes_read_per_query": 462528.0, + "depth": 50, + "doc_recall_at_k": 0.8178888888888889, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.698216294692743, + "p10_recall_at_k": 0.9, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.9590000000000003, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 539328.0, + "depth": 100, + "doc_recall_at_k": 0.8178888888888889, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.6981681784223372, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.9920000000000001, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 426590.72, + "depth": 27, + "doc_recall_at_k": 0.8178888888888889, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 26.60333333333333, + "metric": "cosine", + "ndcg_at_k": 0.6985308686358509, + "p10_recall_at_k": 0.8, + "p90_depth": 39, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 300, + "recall_at_k": 0.8890000000000019, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9966666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 492526.08, + "depth": 70, + "doc_recall_at_k": 0.8178888888888889, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 69.53, + "metric": "cosine", + "ndcg_at_k": 0.6981681784223372, + "p10_recall_at_k": 0.9, + "p90_depth": 119, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 300, + "recall_at_k": 0.9850000000000005, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 649623.04, + "depth": 172, + "doc_recall_at_k": 0.8178888888888889, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 171.80666666666667, + "metric": "cosine", + "ndcg_at_k": 0.698128294524517, + "p10_recall_at_k": 1.0, + "p90_depth": 312, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 300, + "recall_at_k": 0.999, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 385728.0, + "depth": 0, + "doc_recall_at_k": 0.7895555555555556, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.6641970292909967, + "p10_recall_at_k": 0.5, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 300, + "recall_at_k": 0.6939999999999998, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.74, + "vectors": 8036 + }, + { + "bytes_read_per_query": 393568.0, + "depth": 20, + "doc_recall_at_k": 0.8012222222222223, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.6926358803755148, + "p10_recall_at_k": 0.7, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.8793333333333343, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9966666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 405328.0, + "depth": 50, + "doc_recall_at_k": 0.8178888888888889, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.6979826756558765, + "p10_recall_at_k": 0.9, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.9726666666666672, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9966666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 424928.0, + "depth": 100, + "doc_recall_at_k": 0.8178888888888889, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.6978365964864137, + "p10_recall_at_k": 0.9, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.9850000000000004, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9966666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 393854.16, + "depth": 21, + "doc_recall_at_k": 0.8068888888888889, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 20.73, + "metric": "cosine", + "ndcg_at_k": 0.6946942588586353, + "p10_recall_at_k": 0.8, + "p90_depth": 30, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 300, + "recall_at_k": 0.8903333333333345, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9966666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 402380.16, + "depth": 42, + "doc_recall_at_k": 0.8178888888888889, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 42.48, + "metric": "cosine", + "ndcg_at_k": 0.6979826756558765, + "p10_recall_at_k": 0.9, + "p90_depth": 68, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 300, + "recall_at_k": 0.9716666666666675, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9966666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 418797.12, + "depth": 84, + "doc_recall_at_k": 0.8178888888888889, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 84.36, + "metric": "cosine", + "ndcg_at_k": 0.6978365964864137, + "p10_recall_at_k": 0.9, + "p90_depth": 152, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 300, + "recall_at_k": 0.9850000000000002, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9966666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 416448.0, + "depth": 20, + "doc_recall_at_k": 0.8012222222222223, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.6925389758529803, + "p10_recall_at_k": 0.7, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.881000000000001, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 462528.0, + "depth": 50, + "doc_recall_at_k": 0.8178888888888889, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.6982743736939797, + "p10_recall_at_k": 0.9, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.9840000000000001, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 539328.0, + "depth": 100, + "doc_recall_at_k": 0.8178888888888889, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.698128294524517, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.9980000000000001, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 417569.28, + "depth": 21, + "doc_recall_at_k": 0.8035555555555556, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 20.73, + "metric": "cosine", + "ndcg_at_k": 0.6935856886446354, + "p10_recall_at_k": 0.8, + "p90_depth": 30, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 300, + "recall_at_k": 0.8923333333333344, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 450977.28, + "depth": 42, + "doc_recall_at_k": 0.8178888888888889, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 42.48, + "metric": "cosine", + "ndcg_at_k": 0.6982743736939797, + "p10_recall_at_k": 0.9, + "p90_depth": 68, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 300, + "recall_at_k": 0.9823333333333338, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 515304.95999999996, + "depth": 84, + "doc_recall_at_k": 0.8178888888888889, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 84.36, + "metric": "cosine", + "ndcg_at_k": 0.698128294524517, + "p10_recall_at_k": 1.0, + "p90_depth": 152, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 300, + "recall_at_k": 0.9973333333333333, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 385728.0, + "depth": 0, + "doc_recall_at_k": 0.7864444444444444, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.661924840051994, + "p10_recall_at_k": 0.5, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 300, + "recall_at_k": 0.7029999999999993, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.78, + "vectors": 8036 + }, + { + "bytes_read_per_query": 393568.0, + "depth": 20, + "doc_recall_at_k": 0.8112222222222223, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.6957615849020348, + "p10_recall_at_k": 0.7, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.8933333333333349, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9966666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 405328.0, + "depth": 50, + "doc_recall_at_k": 0.8178888888888889, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.6978365964864137, + "p10_recall_at_k": 0.9, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.9790000000000003, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9966666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 424928.0, + "depth": 100, + "doc_recall_at_k": 0.8178888888888889, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.6978365964864137, + "p10_recall_at_k": 0.9, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.9863333333333336, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9966666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 393914.26666666666, + "depth": 21, + "doc_recall_at_k": 0.8078888888888889, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 20.883333333333333, + "metric": "cosine", + "ndcg_at_k": 0.6941037879793005, + "p10_recall_at_k": 0.8, + "p90_depth": 29, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 300, + "recall_at_k": 0.910666666666668, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9966666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 402952.48, + "depth": 44, + "doc_recall_at_k": 0.8145555555555556, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 43.94, + "metric": "cosine", + "ndcg_at_k": 0.6967254853753027, + "p10_recall_at_k": 0.9, + "p90_depth": 69, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 300, + "recall_at_k": 0.976666666666667, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9966666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 419880.3466666667, + "depth": 87, + "doc_recall_at_k": 0.8178888888888889, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 87.12333333333333, + "metric": "cosine", + "ndcg_at_k": 0.6978365964864137, + "p10_recall_at_k": 0.9, + "p90_depth": 151, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 300, + "recall_at_k": 0.9860000000000002, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9966666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 416448.0, + "depth": 20, + "doc_recall_at_k": 0.8112222222222223, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.6960418376887184, + "p10_recall_at_k": 0.7, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.8963333333333348, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 462528.0, + "depth": 50, + "doc_recall_at_k": 0.8178888888888889, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.698128294524517, + "p10_recall_at_k": 1.0, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.9900000000000002, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 539328.0, + "depth": 100, + "doc_recall_at_k": 0.8178888888888889, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.698128294524517, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.9990000000000001, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 417804.8, + "depth": 21, + "doc_recall_at_k": 0.8078888888888889, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 20.883333333333333, + "metric": "cosine", + "ndcg_at_k": 0.6943840407659843, + "p10_recall_at_k": 0.8, + "p90_depth": 29, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 300, + "recall_at_k": 0.9130000000000013, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 453219.83999999997, + "depth": 44, + "doc_recall_at_k": 0.8145555555555556, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 43.94, + "metric": "cosine", + "ndcg_at_k": 0.6970171834134059, + "p10_recall_at_k": 0.9, + "p90_depth": 69, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 300, + "recall_at_k": 0.9880000000000001, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 519549.44, + "depth": 87, + "doc_recall_at_k": 0.8178888888888889, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 87.12333333333333, + "metric": "cosine", + "ndcg_at_k": 0.698128294524517, + "p10_recall_at_k": 1.0, + "p90_depth": 151, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 300, + "recall_at_k": 0.999, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 192864.0, + "depth": 0, + "doc_recall_at_k": 0.7254444444444444, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 300, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.6021298615038453, + "p10_recall_at_k": 0.4, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 300, + "recall_at_k": 0.5766666666666663, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.6233333333333333, + "vectors": 8036 + }, + { + "bytes_read_per_query": 200704.0, + "depth": 20, + "doc_recall_at_k": 0.7884444444444444, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 300, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.6787604975161192, + "p10_recall_at_k": 0.5, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.7626666666666668, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9633333333333334, + "vectors": 8036 + }, + { + "bytes_read_per_query": 212464.0, + "depth": 50, + "doc_recall_at_k": 0.8107777777777777, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 300, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.6936244935556362, + "p10_recall_at_k": 0.8, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.9096666666666677, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9833333333333333, + "vectors": 8036 + }, + { + "bytes_read_per_query": 232064.0, + "depth": 100, + "doc_recall_at_k": 0.8145555555555556, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 300, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.6968730470653541, + "p10_recall_at_k": 0.9, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.9616666666666669, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9933333333333333, + "vectors": 8036 + }, + { + "bytes_read_per_query": 208022.64, + "depth": 39, + "doc_recall_at_k": 0.8101111111111111, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 300, + "mean_depth": 38.67, + "metric": "cosine", + "ndcg_at_k": 0.6929082905574537, + "p10_recall_at_k": 0.7, + "p90_depth": 63, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 300, + "recall_at_k": 0.8836666666666685, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9866666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 244507.38666666666, + "depth": 132, + "doc_recall_at_k": 0.8185555555555556, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 300, + "mean_depth": 131.74333333333334, + "metric": "cosine", + "ndcg_at_k": 0.6982250374072293, + "p10_recall_at_k": 0.9, + "p90_depth": 249, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 300, + "recall_at_k": 0.9773333333333333, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9966666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 333371.17333333334, + "depth": 358, + "doc_recall_at_k": 0.8178888888888889, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 300, + "mean_depth": 358.43666666666667, + "metric": "cosine", + "ndcg_at_k": 0.6978365964864137, + "p10_recall_at_k": 0.9, + "p90_depth": 722, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 300, + "recall_at_k": 0.9863333333333336, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9966666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 223584.0, + "depth": 20, + "doc_recall_at_k": 0.7851111111111111, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 300, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.6779906832341057, + "p10_recall_at_k": 0.5, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.7630000000000002, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9666666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 269664.0, + "depth": 50, + "doc_recall_at_k": 0.8107777777777777, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 300, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.6939161915937393, + "p10_recall_at_k": 0.8, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.9136666666666677, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9866666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 346464.0, + "depth": 100, + "doc_recall_at_k": 0.8145555555555556, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 300, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.6971647451034574, + "p10_recall_at_k": 0.9, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.9703333333333334, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9966666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 252261.12, + "depth": 39, + "doc_recall_at_k": 0.8101111111111111, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 300, + "mean_depth": 38.67, + "metric": "cosine", + "ndcg_at_k": 0.6931999885955569, + "p10_recall_at_k": 0.7, + "p90_depth": 63, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 300, + "recall_at_k": 0.8860000000000016, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.99, + "vectors": 8036 + }, + { + "bytes_read_per_query": 395221.76, + "depth": 132, + "doc_recall_at_k": 0.8185555555555556, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 300, + "mean_depth": 131.74333333333334, + "metric": "cosine", + "ndcg_at_k": 0.6985167354453325, + "p10_recall_at_k": 0.9, + "p90_depth": 249, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 300, + "recall_at_k": 0.9886666666666664, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 743422.72, + "depth": 358, + "doc_recall_at_k": 0.8178888888888889, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 300, + "mean_depth": 358.43666666666667, + "metric": "cosine", + "ndcg_at_k": 0.698128294524517, + "p10_recall_at_k": 1.0, + "p90_depth": 722, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 300, + "recall_at_k": 0.9993333333333334, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 192864.0, + "depth": 0, + "doc_recall_at_k": 0.740111111111111, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 300, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.595462153921237, + "p10_recall_at_k": 0.4, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 300, + "recall_at_k": 0.5809999999999995, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.6233333333333333, + "vectors": 8036 + }, + { + "bytes_read_per_query": 200704.0, + "depth": 20, + "doc_recall_at_k": 0.7662222222222222, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 300, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.6739720466578167, + "p10_recall_at_k": 0.5, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.7640000000000001, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9633333333333334, + "vectors": 8036 + }, + { + "bytes_read_per_query": 212464.0, + "depth": 50, + "doc_recall_at_k": 0.7945555555555556, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 300, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.6858115123268185, + "p10_recall_at_k": 0.8, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.9130000000000013, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9866666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 232064.0, + "depth": 100, + "doc_recall_at_k": 0.8112222222222223, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 300, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.6944399447458396, + "p10_recall_at_k": 0.9, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.9603333333333336, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9933333333333333, + "vectors": 8036 + }, + { + "bytes_read_per_query": 205530.82666666666, + "depth": 32, + "doc_recall_at_k": 0.7912222222222223, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 300, + "mean_depth": 32.31333333333333, + "metric": "cosine", + "ndcg_at_k": 0.6869098397436086, + "p10_recall_at_k": 0.7, + "p90_depth": 49, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 300, + "recall_at_k": 0.8550000000000019, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9833333333333333, + "vectors": 8036 + }, + { + "bytes_read_per_query": 229517.30666666667, + "depth": 94, + "doc_recall_at_k": 0.8112222222222223, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 300, + "mean_depth": 93.50333333333333, + "metric": "cosine", + "ndcg_at_k": 0.6949057555878569, + "p10_recall_at_k": 0.9, + "p90_depth": 154, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 300, + "recall_at_k": 0.9676666666666671, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9966666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 285002.29333333333, + "depth": 235, + "doc_recall_at_k": 0.8178888888888889, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 300, + "mean_depth": 235.04666666666665, + "metric": "cosine", + "ndcg_at_k": 0.6978365964864137, + "p10_recall_at_k": 0.9, + "p90_depth": 410, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 300, + "recall_at_k": 0.9840000000000003, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9966666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 223584.0, + "depth": 20, + "doc_recall_at_k": 0.7662222222222222, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 300, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.6738751421352822, + "p10_recall_at_k": 0.5, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.7646666666666668, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9666666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 269664.0, + "depth": 50, + "doc_recall_at_k": 0.7945555555555556, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 300, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.6856664915338782, + "p10_recall_at_k": 0.8, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.9173333333333344, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.99, + "vectors": 8036 + }, + { + "bytes_read_per_query": 346464.0, + "depth": 100, + "doc_recall_at_k": 0.8112222222222223, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 300, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.6947316427839428, + "p10_recall_at_k": 0.9, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.9686666666666668, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9966666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 242497.28, + "depth": 32, + "doc_recall_at_k": 0.7912222222222223, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 300, + "mean_depth": 32.31333333333333, + "metric": "cosine", + "ndcg_at_k": 0.6867648189506682, + "p10_recall_at_k": 0.7, + "p90_depth": 49, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 300, + "recall_at_k": 0.857333333333335, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9866666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 336485.12, + "depth": 94, + "doc_recall_at_k": 0.8112222222222223, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 300, + "mean_depth": 93.50333333333333, + "metric": "cosine", + "ndcg_at_k": 0.6951378921041349, + "p10_recall_at_k": 0.9, + "p90_depth": 154, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 300, + "recall_at_k": 0.9760000000000001, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 553895.6799999999, + "depth": 235, + "doc_recall_at_k": 0.8178888888888889, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 300, + "mean_depth": 235.04666666666665, + "metric": "cosine", + "ndcg_at_k": 0.698128294524517, + "p10_recall_at_k": 1.0, + "p90_depth": 410, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 300, + "recall_at_k": 0.9970000000000001, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 192864.0, + "depth": 0, + "doc_recall_at_k": 0.7317777777777777, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 300, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.598578404548837, + "p10_recall_at_k": 0.3, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 300, + "recall_at_k": 0.564, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.6333333333333333, + "vectors": 8036 + }, + { + "bytes_read_per_query": 200704.0, + "depth": 20, + "doc_recall_at_k": 0.7562222222222222, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 300, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.6671410172784769, + "p10_recall_at_k": 0.5, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.759, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.96, + "vectors": 8036 + }, + { + "bytes_read_per_query": 212464.0, + "depth": 50, + "doc_recall_at_k": 0.7912222222222223, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 300, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.6857666977605519, + "p10_recall_at_k": 0.8, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.9123333333333347, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9933333333333333, + "vectors": 8036 + }, + { + "bytes_read_per_query": 232064.0, + "depth": 100, + "doc_recall_at_k": 0.8045555555555556, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 300, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.6909053002449109, + "p10_recall_at_k": 0.9, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.964666666666667, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9966666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 206469.01333333334, + "depth": 35, + "doc_recall_at_k": 0.7901111111111111, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 300, + "mean_depth": 34.70666666666666, + "metric": "cosine", + "ndcg_at_k": 0.6852695837825471, + "p10_recall_at_k": 0.7, + "p90_depth": 52, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 300, + "recall_at_k": 0.872666666666668, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.99, + "vectors": 8036 + }, + { + "bytes_read_per_query": 233763.97333333333, + "depth": 104, + "doc_recall_at_k": 0.8178888888888889, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 300, + "mean_depth": 104.33666666666667, + "metric": "cosine", + "ndcg_at_k": 0.6979442742786449, + "p10_recall_at_k": 0.9, + "p90_depth": 176, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 300, + "recall_at_k": 0.9723333333333338, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9966666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 295339.3333333334, + "depth": 261, + "doc_recall_at_k": 0.8178888888888889, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 300, + "mean_depth": 261.4166666666667, + "metric": "cosine", + "ndcg_at_k": 0.6978365964864137, + "p10_recall_at_k": 0.9, + "p90_depth": 479, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 300, + "recall_at_k": 0.985666666666667, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9966666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 223584.0, + "depth": 20, + "doc_recall_at_k": 0.7528888888888889, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 300, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.6660324470644768, + "p10_recall_at_k": 0.5, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.7593333333333334, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9633333333333334, + "vectors": 8036 + }, + { + "bytes_read_per_query": 269664.0, + "depth": 50, + "doc_recall_at_k": 0.7912222222222223, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 300, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.6859988342768296, + "p10_recall_at_k": 0.8, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.9173333333333347, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9966666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 346464.0, + "depth": 100, + "doc_recall_at_k": 0.8045555555555556, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 300, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.6911969982830141, + "p10_recall_at_k": 0.9, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.9743333333333334, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 246173.44, + "depth": 35, + "doc_recall_at_k": 0.7901111111111111, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 300, + "mean_depth": 34.70666666666666, + "metric": "cosine", + "ndcg_at_k": 0.685355641129362, + "p10_recall_at_k": 0.7, + "p90_depth": 52, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 300, + "recall_at_k": 0.8743333333333347, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9933333333333333, + "vectors": 8036 + }, + { + "bytes_read_per_query": 353125.12, + "depth": 104, + "doc_recall_at_k": 0.8178888888888889, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 300, + "mean_depth": 104.33666666666667, + "metric": "cosine", + "ndcg_at_k": 0.6981764107949228, + "p10_recall_at_k": 0.9, + "p90_depth": 176, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 300, + "recall_at_k": 0.9840000000000003, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 594400.0, + "depth": 261, + "doc_recall_at_k": 0.8178888888888889, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 300, + "mean_depth": 261.4166666666667, + "metric": "cosine", + "ndcg_at_k": 0.698128294524517, + "p10_recall_at_k": 1.0, + "p90_depth": 479, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 300, + "recall_at_k": 0.9986666666666667, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 771456.0, + "depth": 0, + "doc_recall_at_k": 0.8168888888888889, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 300, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.6918877322341538, + "p10_recall_at_k": 0.7, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 300, + "recall_at_k": 0.8446666666666696, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.8866666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 779296.0, + "depth": 20, + "doc_recall_at_k": 0.8178888888888889, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 300, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.6978365964864137, + "p10_recall_at_k": 0.9, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.9753333333333338, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9966666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 791056.0, + "depth": 50, + "doc_recall_at_k": 0.8178888888888889, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 300, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.6978365964864137, + "p10_recall_at_k": 0.9, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.9870000000000002, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9966666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 810656.0, + "depth": 100, + "doc_recall_at_k": 0.8178888888888889, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 300, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.6978365964864137, + "p10_recall_at_k": 0.9, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.9870000000000002, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9966666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 776932.24, + "depth": 14, + "doc_recall_at_k": 0.8112222222222223, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 300, + "mean_depth": 13.97, + "metric": "cosine", + "ndcg_at_k": 0.6958100522246489, + "p10_recall_at_k": 0.9, + "p90_depth": 17, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 300, + "recall_at_k": 0.9490000000000011, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9933333333333333, + "vectors": 8036 + }, + { + "bytes_read_per_query": 779107.84, + "depth": 20, + "doc_recall_at_k": 0.8178888888888889, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 300, + "mean_depth": 19.52, + "metric": "cosine", + "ndcg_at_k": 0.6978365964864137, + "p10_recall_at_k": 0.9, + "p90_depth": 27, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 300, + "recall_at_k": 0.9783333333333337, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9966666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 782145.84, + "depth": 27, + "doc_recall_at_k": 0.8178888888888889, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 300, + "mean_depth": 27.27, + "metric": "cosine", + "ndcg_at_k": 0.6978365964864137, + "p10_recall_at_k": 0.9, + "p90_depth": 41, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 300, + "recall_at_k": 0.9870000000000001, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9966666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 802176.0, + "depth": 20, + "doc_recall_at_k": 0.8178888888888889, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 300, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.698128294524517, + "p10_recall_at_k": 0.9, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.985666666666667, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 848256.0, + "depth": 50, + "doc_recall_at_k": 0.8178888888888889, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 300, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.698128294524517, + "p10_recall_at_k": 1.0, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 1.0, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 925056.0, + "depth": 100, + "doc_recall_at_k": 0.8178888888888889, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 300, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.698128294524517, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 1.0, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 792913.92, + "depth": 14, + "doc_recall_at_k": 0.8112222222222223, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 300, + "mean_depth": 13.97, + "metric": "cosine", + "ndcg_at_k": 0.696101750262752, + "p10_recall_at_k": 0.9, + "p90_depth": 17, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 300, + "recall_at_k": 0.9550000000000013, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9966666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 801438.72, + "depth": 20, + "doc_recall_at_k": 0.8178888888888889, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 300, + "mean_depth": 19.52, + "metric": "cosine", + "ndcg_at_k": 0.698128294524517, + "p10_recall_at_k": 0.9, + "p90_depth": 27, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 300, + "recall_at_k": 0.9886666666666669, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 813342.72, + "depth": 27, + "doc_recall_at_k": 0.8178888888888889, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 300, + "mean_depth": 27.27, + "metric": "cosine", + "ndcg_at_k": 0.698128294524517, + "p10_recall_at_k": 1.0, + "p90_depth": 41, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 300, + "recall_at_k": 0.9993333333333334, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 771456.0, + "depth": 0, + "doc_recall_at_k": 0.8178888888888889, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 300, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.6910598965004005, + "p10_recall_at_k": 0.7, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 300, + "recall_at_k": 0.8430000000000023, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.91, + "vectors": 8036 + }, + { + "bytes_read_per_query": 779296.0, + "depth": 20, + "doc_recall_at_k": 0.8112222222222223, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 300, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.6957619359542431, + "p10_recall_at_k": 0.9, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.9736666666666669, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9966666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 791056.0, + "depth": 50, + "doc_recall_at_k": 0.8178888888888889, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 300, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.6978365964864137, + "p10_recall_at_k": 0.9, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.9870000000000002, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9966666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 810656.0, + "depth": 100, + "doc_recall_at_k": 0.8178888888888889, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 300, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.6978365964864137, + "p10_recall_at_k": 0.9, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.9870000000000002, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9966666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 776968.8266666667, + "depth": 14, + "doc_recall_at_k": 0.8112222222222223, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 300, + "mean_depth": 14.063333333333333, + "metric": "cosine", + "ndcg_at_k": 0.6956739357860169, + "p10_recall_at_k": 0.9, + "p90_depth": 18, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 300, + "recall_at_k": 0.9456666666666674, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9966666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 779345.6533333333, + "depth": 20, + "doc_recall_at_k": 0.8112222222222223, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 300, + "mean_depth": 20.126666666666665, + "metric": "cosine", + "ndcg_at_k": 0.6957619359542431, + "p10_recall_at_k": 0.9, + "p90_depth": 29, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 300, + "recall_at_k": 0.9783333333333335, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9966666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 782604.48, + "depth": 28, + "doc_recall_at_k": 0.8178888888888889, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 300, + "mean_depth": 28.44, + "metric": "cosine", + "ndcg_at_k": 0.6978365964864137, + "p10_recall_at_k": 0.9, + "p90_depth": 43, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 300, + "recall_at_k": 0.9863333333333336, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9966666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 802176.0, + "depth": 20, + "doc_recall_at_k": 0.8112222222222223, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 300, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.6960536339923463, + "p10_recall_at_k": 0.9, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.9833333333333335, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 848256.0, + "depth": 50, + "doc_recall_at_k": 0.8178888888888889, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 300, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.698128294524517, + "p10_recall_at_k": 1.0, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 1.0, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 925056.0, + "depth": 100, + "doc_recall_at_k": 0.8178888888888889, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 300, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.698128294524517, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 1.0, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 793057.28, + "depth": 14, + "doc_recall_at_k": 0.8112222222222223, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 300, + "mean_depth": 14.063333333333333, + "metric": "cosine", + "ndcg_at_k": 0.6959060723022947, + "p10_recall_at_k": 0.9, + "p90_depth": 18, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 300, + "recall_at_k": 0.9493333333333339, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 802370.56, + "depth": 20, + "doc_recall_at_k": 0.8112222222222223, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 300, + "mean_depth": 20.126666666666665, + "metric": "cosine", + "ndcg_at_k": 0.6960536339923463, + "p10_recall_at_k": 1.0, + "p90_depth": 29, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 300, + "recall_at_k": 0.99, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 815139.84, + "depth": 28, + "doc_recall_at_k": 0.8178888888888889, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 300, + "mean_depth": 28.44, + "metric": "cosine", + "ndcg_at_k": 0.698128294524517, + "p10_recall_at_k": 1.0, + "p90_depth": 43, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 300, + "recall_at_k": 0.9993333333333334, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + } + ], + "profiles": [ + { + "depth_for_recall": [ + [ + 0.9, + 40 + ], + [ + 0.95, + 71 + ], + [ + 0.99, + 192 + ], + [ + 1.0, + 633 + ] + ], + "first": "binary", + "first_bytes": 48, + "k": 10, + "median_full_depth": 45.0, + "metric": "cosine", + "neighbour_band": 0.013464927673339844, + "queries": 300, + "rank_noise": 0.018715791052223443, + "recall_at_k": 0.6109999999999995, + "vectors": 8036 + }, + { + "depth_for_recall": [ + [ + 0.9, + 30 + ], + [ + 0.95, + 45 + ], + [ + 0.99, + 90 + ], + [ + 1.0, + 441 + ] + ], + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "median_full_depth": 33.0, + "metric": "cosine", + "neighbour_band": 0.013464927673339844, + "queries": 300, + "rank_noise": 0.013085613849637225, + "recall_at_k": 0.6559999999999993, + "vectors": 8036 + }, + { + "depth_for_recall": [ + [ + 0.9, + 21 + ], + [ + 0.95, + 31 + ], + [ + 0.99, + 59 + ], + [ + 1.0, + 271 + ] + ], + "first": "pq48", + "first_bytes": 48, + "k": 10, + "median_full_depth": 23.0, + "metric": "cosine", + "neighbour_band": 0.013464927673339844, + "queries": 300, + "rank_noise": 0.011187321269100204, + "recall_at_k": 0.711333333333333, + "vectors": 8036 + }, + { + "depth_for_recall": [ + [ + 0.9, + 30 + ], + [ + 0.95, + 45 + ], + [ + 0.99, + 93 + ], + [ + 1.0, + 497 + ] + ], + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "median_full_depth": 33.5, + "metric": "cosine", + "neighbour_band": 0.013464927673339844, + "queries": 300, + "rank_noise": 0.013221591591441402, + "recall_at_k": 0.6533333333333328, + "vectors": 8036 + }, + { + "depth_for_recall": [ + [ + 0.9, + 23 + ], + [ + 0.95, + 31 + ], + [ + 0.99, + 61 + ], + [ + 1.0, + 292 + ] + ], + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "median_full_depth": 25.0, + "metric": "cosine", + "neighbour_band": 0.013464927673339844, + "queries": 300, + "rank_noise": 0.011861118379885908, + "recall_at_k": 0.6939999999999998, + "vectors": 8036 + }, + { + "depth_for_recall": [ + [ + 0.9, + 21 + ], + [ + 0.95, + 28 + ], + [ + 0.99, + 50 + ], + [ + 1.0, + 154 + ] + ], + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "median_full_depth": 23.0, + "metric": "cosine", + "neighbour_band": 0.013464927673339844, + "queries": 300, + "rank_noise": 0.011779424581963384, + "recall_at_k": 0.7029999999999993, + "vectors": 8036 + }, + { + "depth_for_recall": [ + [ + 0.9, + 45 + ], + [ + 0.95, + 71 + ], + [ + 0.99, + 183 + ], + [ + 1.0, + 601 + ] + ], + "first": "pq24", + "first_bytes": 24, + "k": 10, + "median_full_depth": 50.0, + "metric": "cosine", + "neighbour_band": 0.013464927673339844, + "queries": 300, + "rank_noise": 0.01640890269556495, + "recall_at_k": 0.5766666666666663, + "vectors": 8036 + }, + { + "depth_for_recall": [ + [ + 0.9, + 44 + ], + [ + 0.95, + 72 + ], + [ + 0.99, + 162 + ], + [ + 1.0, + 732 + ] + ], + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "median_full_depth": 47.0, + "metric": "cosine", + "neighbour_band": 0.013464927673339844, + "queries": 300, + "rank_noise": 0.01636725122453803, + "recall_at_k": 0.5809999999999995, + "vectors": 8036 + }, + { + "depth_for_recall": [ + [ + 0.9, + 43 + ], + [ + 0.95, + 71 + ], + [ + 0.99, + 173 + ], + [ + 1.0, + 832 + ] + ], + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "median_full_depth": 45.0, + "metric": "cosine", + "neighbour_band": 0.013464927673339844, + "queries": 300, + "rank_noise": 0.01673124351502799, + "recall_at_k": 0.564, + "vectors": 8036 + }, + { + "depth_for_recall": [ + [ + 0.9, + 12 + ], + [ + 0.95, + 15 + ], + [ + 0.99, + 22 + ], + [ + 1.0, + 37 + ] + ], + "first": "pq96", + "first_bytes": 96, + "k": 10, + "median_full_depth": 13.0, + "metric": "cosine", + "neighbour_band": 0.013464927673339844, + "queries": 300, + "rank_noise": 0.0059490096574797815, + "recall_at_k": 0.8446666666666696, + "vectors": 8036 + }, + { + "depth_for_recall": [ + [ + 0.9, + 12 + ], + [ + 0.95, + 15 + ], + [ + 0.99, + 23 + ], + [ + 1.0, + 40 + ] + ], + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "median_full_depth": 13.0, + "metric": "cosine", + "neighbour_band": 0.013464927673339844, + "queries": 300, + "rank_noise": 0.006265162117005945, + "recall_at_k": 0.8430000000000023, + "vectors": 8036 + } + ] +} diff --git a/experiments/2026-11-hybrid-adaptive-matrix/results/scifact-minilm-k10.json b/experiments/2026-11-hybrid-adaptive-matrix/results/scifact-minilm-k10.json new file mode 100644 index 0000000..b93c1ff --- /dev/null +++ b/experiments/2026-11-hybrid-adaptive-matrix/results/scifact-minilm-k10.json @@ -0,0 +1,3900 @@ +{ + "cells": [ + { + "bytes_read_per_query": 385728.0, + "depth": 0, + "doc_recall_at_k": 0.7334444444444445, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.604971709997024, + "p10_recall_at_k": 0.5, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 300, + "recall_at_k": 0.7049999999999998, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.73, + "vectors": 8036 + }, + { + "bytes_read_per_query": 393568.0, + "depth": 20, + "doc_recall_at_k": 0.762888888888889, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.6226293323459402, + "p10_recall_at_k": 0.7, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.8770000000000012, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9966666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 405328.0, + "depth": 50, + "doc_recall_at_k": 0.777888888888889, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.628117302061305, + "p10_recall_at_k": 0.9, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.9680000000000005, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9966666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 424928.0, + "depth": 100, + "doc_recall_at_k": 0.777888888888889, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.6267937784557772, + "p10_recall_at_k": 0.9, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.9883333333333333, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 394431.70666666667, + "depth": 22, + "doc_recall_at_k": 0.7706666666666667, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 22.203333333333333, + "metric": "cosine", + "ndcg_at_k": 0.6248485359720273, + "p10_recall_at_k": 0.8, + "p90_depth": 32, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 300, + "recall_at_k": 0.9033333333333354, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9966666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 405567.12, + "depth": 51, + "doc_recall_at_k": 0.777888888888889, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 50.61, + "metric": "cosine", + "ndcg_at_k": 0.6268275063847177, + "p10_recall_at_k": 0.9, + "p90_depth": 84, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 300, + "recall_at_k": 0.9803333333333332, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 429429.4666666667, + "depth": 111, + "doc_recall_at_k": 0.777888888888889, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 111.48333333333333, + "metric": "cosine", + "ndcg_at_k": 0.6266916263886889, + "p10_recall_at_k": 1.0, + "p90_depth": 199, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 300, + "recall_at_k": 0.9940000000000001, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 416448.0, + "depth": 20, + "doc_recall_at_k": 0.762888888888889, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.6225443333756476, + "p10_recall_at_k": 0.7, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.8770000000000012, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9966666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 462528.0, + "depth": 50, + "doc_recall_at_k": 0.777888888888889, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.6282633812307677, + "p10_recall_at_k": 0.9, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.9706666666666672, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9966666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 539328.0, + "depth": 100, + "doc_recall_at_k": 0.777888888888889, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.6269398576252401, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.9926666666666667, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 419832.32, + "depth": 22, + "doc_recall_at_k": 0.7706666666666667, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 22.203333333333333, + "metric": "cosine", + "ndcg_at_k": 0.6247635370017347, + "p10_recall_at_k": 0.8, + "p90_depth": 32, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 300, + "recall_at_k": 0.9040000000000018, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9966666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 463464.95999999996, + "depth": 51, + "doc_recall_at_k": 0.777888888888889, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 50.61, + "metric": "cosine", + "ndcg_at_k": 0.6269735855541806, + "p10_recall_at_k": 0.9, + "p90_depth": 84, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 300, + "recall_at_k": 0.9830000000000001, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 556966.4, + "depth": 111, + "doc_recall_at_k": 0.777888888888889, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 111.48333333333333, + "metric": "cosine", + "ndcg_at_k": 0.6267937784557772, + "p10_recall_at_k": 1.0, + "p90_depth": 199, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 300, + "recall_at_k": 0.9986666666666667, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 385728.0, + "depth": 0, + "doc_recall_at_k": 0.7387777777777778, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.597754249570433, + "p10_recall_at_k": 0.5, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 300, + "recall_at_k": 0.718666666666667, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.7166666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 393568.0, + "depth": 20, + "doc_recall_at_k": 0.7806666666666667, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.628959126523926, + "p10_recall_at_k": 0.7, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.8986666666666681, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.99, + "vectors": 8036 + }, + { + "bytes_read_per_query": 405328.0, + "depth": 50, + "doc_recall_at_k": 0.7745555555555556, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.6265458335383647, + "p10_recall_at_k": 0.9, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.9790000000000003, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9966666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 424928.0, + "depth": 100, + "doc_recall_at_k": 0.777888888888889, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.6279814220652761, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.9929999999999999, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9966666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 394433.0133333333, + "depth": 22, + "doc_recall_at_k": 0.774, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 22.206666666666667, + "metric": "cosine", + "ndcg_at_k": 0.6264031688069849, + "p10_recall_at_k": 0.8, + "p90_depth": 32, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 300, + "recall_at_k": 0.9196666666666689, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9933333333333333, + "vectors": 8036 + }, + { + "bytes_read_per_query": 405483.49333333335, + "depth": 50, + "doc_recall_at_k": 0.777888888888889, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 50.39666666666667, + "metric": "cosine", + "ndcg_at_k": 0.6280151499942166, + "p10_recall_at_k": 0.9, + "p90_depth": 84, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 300, + "recall_at_k": 0.9883333333333333, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9966666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 428857.14666666667, + "depth": 110, + "doc_recall_at_k": 0.777888888888889, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 110.02333333333333, + "metric": "cosine", + "ndcg_at_k": 0.6266916263886889, + "p10_recall_at_k": 1.0, + "p90_depth": 199, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 300, + "recall_at_k": 0.9946666666666666, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 416448.0, + "depth": 20, + "doc_recall_at_k": 0.7773333333333334, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.6279105781325739, + "p10_recall_at_k": 0.7, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.8993333333333347, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.99, + "vectors": 8036 + }, + { + "bytes_read_per_query": 462528.0, + "depth": 50, + "doc_recall_at_k": 0.7745555555555556, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.6264169074656978, + "p10_recall_at_k": 0.9, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.9820000000000004, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9966666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 539328.0, + "depth": 100, + "doc_recall_at_k": 0.777888888888889, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.6280835741323644, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.9973333333333331, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9966666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 419837.44, + "depth": 22, + "doc_recall_at_k": 0.774, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 22.206666666666667, + "metric": "cosine", + "ndcg_at_k": 0.6263181698366923, + "p10_recall_at_k": 0.8, + "p90_depth": 32, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 300, + "recall_at_k": 0.9206666666666687, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9933333333333333, + "vectors": 8036 + }, + { + "bytes_read_per_query": 463137.28, + "depth": 50, + "doc_recall_at_k": 0.777888888888889, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 50.39666666666667, + "metric": "cosine", + "ndcg_at_k": 0.628117302061305, + "p10_recall_at_k": 1.0, + "p90_depth": 84, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 300, + "recall_at_k": 0.9923333333333331, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9966666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 554723.84, + "depth": 110, + "doc_recall_at_k": 0.777888888888889, + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 110.02333333333333, + "metric": "cosine", + "ndcg_at_k": 0.6267937784557772, + "p10_recall_at_k": 1.0, + "p90_depth": 199, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 300, + "recall_at_k": 0.9996666666666666, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 385728.0, + "depth": 0, + "doc_recall_at_k": 0.7615, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.6128831565370856, + "p10_recall_at_k": 0.6, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 300, + "recall_at_k": 0.7640000000000003, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.7633333333333333, + "vectors": 8036 + }, + { + "bytes_read_per_query": 393568.0, + "depth": 20, + "doc_recall_at_k": 0.7745555555555556, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.6256968192435285, + "p10_recall_at_k": 0.8, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.937666666666668, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 405328.0, + "depth": 50, + "doc_recall_at_k": 0.777888888888889, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.6266916263886889, + "p10_recall_at_k": 0.9, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.9879999999999999, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 424928.0, + "depth": 100, + "doc_recall_at_k": 0.777888888888889, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.6266916263886889, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.9939999999999998, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 392871.5466666667, + "depth": 18, + "doc_recall_at_k": 0.7762222222222223, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 18.223333333333333, + "metric": "cosine", + "ndcg_at_k": 0.6261986557386686, + "p10_recall_at_k": 0.8, + "p90_depth": 26, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 300, + "recall_at_k": 0.9240000000000012, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 398904.4266666667, + "depth": 34, + "doc_recall_at_k": 0.7745555555555556, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 33.61333333333334, + "metric": "cosine", + "ndcg_at_k": 0.6255805152775779, + "p10_recall_at_k": 0.9, + "p90_depth": 54, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 300, + "recall_at_k": 0.9819999999999999, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 409655.68, + "depth": 61, + "doc_recall_at_k": 0.777888888888889, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 61.04, + "metric": "cosine", + "ndcg_at_k": 0.6266916263886889, + "p10_recall_at_k": 1.0, + "p90_depth": 107, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 300, + "recall_at_k": 0.9929999999999999, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 416448.0, + "depth": 20, + "doc_recall_at_k": 0.7745555555555556, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.6257989713106168, + "p10_recall_at_k": 0.8, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.9390000000000011, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 462528.0, + "depth": 50, + "doc_recall_at_k": 0.777888888888889, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.6267937784557772, + "p10_recall_at_k": 1.0, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.9923333333333335, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 539328.0, + "depth": 100, + "doc_recall_at_k": 0.777888888888889, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.6267937784557772, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.9986666666666666, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 413719.04, + "depth": 18, + "doc_recall_at_k": 0.7762222222222223, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 18.223333333333333, + "metric": "cosine", + "ndcg_at_k": 0.6261986557386686, + "p10_recall_at_k": 0.8, + "p90_depth": 26, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 300, + "recall_at_k": 0.9250000000000012, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 437358.08, + "depth": 34, + "doc_recall_at_k": 0.7745555555555556, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 33.61333333333334, + "metric": "cosine", + "ndcg_at_k": 0.6256826673446662, + "p10_recall_at_k": 0.9, + "p90_depth": 54, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 300, + "recall_at_k": 0.9856666666666666, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 479485.44, + "depth": 61, + "doc_recall_at_k": 0.777888888888889, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 61.04, + "metric": "cosine", + "ndcg_at_k": 0.6267937784557772, + "p10_recall_at_k": 1.0, + "p90_depth": 107, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 300, + "recall_at_k": 0.9976666666666665, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 385728.0, + "depth": 0, + "doc_recall_at_k": 0.7423888888888888, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.6034769178261316, + "p10_recall_at_k": 0.5, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 300, + "recall_at_k": 0.7176666666666672, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.7033333333333334, + "vectors": 8036 + }, + { + "bytes_read_per_query": 393568.0, + "depth": 20, + "doc_recall_at_k": 0.779, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.6284438193930797, + "p10_recall_at_k": 0.7, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.8993333333333345, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9933333333333333, + "vectors": 8036 + }, + { + "bytes_read_per_query": 405328.0, + "depth": 50, + "doc_recall_at_k": 0.777888888888889, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.6279814220652761, + "p10_recall_at_k": 0.9, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.9790000000000003, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9966666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 424928.0, + "depth": 100, + "doc_recall_at_k": 0.777888888888889, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.6279814220652761, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.9926666666666667, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9966666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 394352.0, + "depth": 22, + "doc_recall_at_k": 0.7756666666666667, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 22.0, + "metric": "cosine", + "ndcg_at_k": 0.6273496219121861, + "p10_recall_at_k": 0.8, + "p90_depth": 31, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 300, + "recall_at_k": 0.9206666666666685, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9933333333333333, + "vectors": 8036 + }, + { + "bytes_read_per_query": 405103.25333333336, + "depth": 49, + "doc_recall_at_k": 0.777888888888889, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 49.42666666666667, + "metric": "cosine", + "ndcg_at_k": 0.6280151499942166, + "p10_recall_at_k": 0.9, + "p90_depth": 78, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 300, + "recall_at_k": 0.9880000000000001, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9966666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 427737.3333333333, + "depth": 107, + "doc_recall_at_k": 0.777888888888889, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 107.16666666666667, + "metric": "cosine", + "ndcg_at_k": 0.6279218605434506, + "p10_recall_at_k": 1.0, + "p90_depth": 191, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 300, + "recall_at_k": 0.9943333333333334, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9966666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 416448.0, + "depth": 20, + "doc_recall_at_k": 0.779, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.6283189365249668, + "p10_recall_at_k": 0.7, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.9003333333333345, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9933333333333333, + "vectors": 8036 + }, + { + "bytes_read_per_query": 462528.0, + "depth": 50, + "doc_recall_at_k": 0.777888888888889, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.6280835741323644, + "p10_recall_at_k": 0.9, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.9820000000000004, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9966666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 539328.0, + "depth": 100, + "doc_recall_at_k": 0.777888888888889, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.6280835741323644, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.9973333333333335, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9966666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 419520.0, + "depth": 22, + "doc_recall_at_k": 0.7756666666666667, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 22.0, + "metric": "cosine", + "ndcg_at_k": 0.6272646229418936, + "p10_recall_at_k": 0.8, + "p90_depth": 31, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 300, + "recall_at_k": 0.9220000000000018, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9933333333333333, + "vectors": 8036 + }, + { + "bytes_read_per_query": 461647.36, + "depth": 49, + "doc_recall_at_k": 0.777888888888889, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 49.42666666666667, + "metric": "cosine", + "ndcg_at_k": 0.628117302061305, + "p10_recall_at_k": 1.0, + "p90_depth": 78, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 300, + "recall_at_k": 0.9913333333333333, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9966666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 550336.0, + "depth": 107, + "doc_recall_at_k": 0.777888888888889, + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 107.16666666666667, + "metric": "cosine", + "ndcg_at_k": 0.6280240126105391, + "p10_recall_at_k": 1.0, + "p90_depth": 191, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 300, + "recall_at_k": 0.9990000000000001, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9966666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 385728.0, + "depth": 0, + "doc_recall_at_k": 0.7518333333333332, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.6087820421773926, + "p10_recall_at_k": 0.6, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 300, + "recall_at_k": 0.7753333333333347, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.7833333333333333, + "vectors": 8036 + }, + { + "bytes_read_per_query": 393568.0, + "depth": 20, + "doc_recall_at_k": 0.7812222222222223, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.6276490198408687, + "p10_recall_at_k": 0.8, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.9456666666666668, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 405328.0, + "depth": 50, + "doc_recall_at_k": 0.777888888888889, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.6266916263886889, + "p10_recall_at_k": 1.0, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.9906666666666666, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 424928.0, + "depth": 100, + "doc_recall_at_k": 0.777888888888889, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.6266916263886889, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.9946666666666666, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 392291.38666666666, + "depth": 17, + "doc_recall_at_k": 0.7797222222222222, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 16.743333333333332, + "metric": "cosine", + "ndcg_at_k": 0.6267997376305644, + "p10_recall_at_k": 0.8, + "p90_depth": 23, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 300, + "recall_at_k": 0.9236666666666673, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 396765.41333333333, + "depth": 28, + "doc_recall_at_k": 0.777888888888889, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 28.156666666666666, + "metric": "cosine", + "ndcg_at_k": 0.6267397426590947, + "p10_recall_at_k": 0.9, + "p90_depth": 41, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 300, + "recall_at_k": 0.9816666666666662, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 404229.0933333333, + "depth": 47, + "doc_recall_at_k": 0.777888888888889, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 47.196666666666665, + "metric": "cosine", + "ndcg_at_k": 0.6266916263886889, + "p10_recall_at_k": 1.0, + "p90_depth": 76, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 300, + "recall_at_k": 0.9936666666666667, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 416448.0, + "depth": 20, + "doc_recall_at_k": 0.7812222222222223, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.627751171907957, + "p10_recall_at_k": 0.8, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.9476666666666669, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 462528.0, + "depth": 50, + "doc_recall_at_k": 0.777888888888889, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.6267937784557772, + "p10_recall_at_k": 1.0, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.9956666666666668, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 539328.0, + "depth": 100, + "doc_recall_at_k": 0.777888888888889, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.6267937784557772, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.9996666666666666, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 411445.76, + "depth": 17, + "doc_recall_at_k": 0.7797222222222222, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 16.743333333333332, + "metric": "cosine", + "ndcg_at_k": 0.6269018896976527, + "p10_recall_at_k": 0.8, + "p90_depth": 23, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 300, + "recall_at_k": 0.9246666666666673, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 428976.64, + "depth": 28, + "doc_recall_at_k": 0.777888888888889, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 28.156666666666666, + "metric": "cosine", + "ndcg_at_k": 0.6268418947261831, + "p10_recall_at_k": 0.9, + "p90_depth": 41, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 300, + "recall_at_k": 0.9856666666666665, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 458222.08, + "depth": 47, + "doc_recall_at_k": 0.777888888888889, + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 47.196666666666665, + "metric": "cosine", + "ndcg_at_k": 0.6267937784557772, + "p10_recall_at_k": 1.0, + "p90_depth": 76, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 300, + "recall_at_k": 0.9986666666666667, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 385728.0, + "depth": 0, + "doc_recall_at_k": 0.7492222222222221, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.5961480446664614, + "p10_recall_at_k": 0.7, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 300, + "recall_at_k": 0.8043333333333348, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.8066666666666666, + "vectors": 8036 + }, + { + "bytes_read_per_query": 393568.0, + "depth": 20, + "doc_recall_at_k": 0.7745555555555556, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.6257219209987495, + "p10_recall_at_k": 0.9, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.9616666666666669, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9966666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 405328.0, + "depth": 50, + "doc_recall_at_k": 0.777888888888889, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.6266916263886889, + "p10_recall_at_k": 1.0, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.994, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 424928.0, + "depth": 100, + "doc_recall_at_k": 0.777888888888889, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.6266916263886889, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.9946666666666666, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 391818.37333333335, + "depth": 16, + "doc_recall_at_k": 0.7723333333333334, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 15.536666666666667, + "metric": "cosine", + "ndcg_at_k": 0.6250950216505998, + "p10_recall_at_k": 0.8, + "p90_depth": 20, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 300, + "recall_at_k": 0.9290000000000009, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9966666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 395137.30666666664, + "depth": 24, + "doc_recall_at_k": 0.7745555555555556, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 24.003333333333334, + "metric": "cosine", + "ndcg_at_k": 0.6256881930698088, + "p10_recall_at_k": 0.9, + "p90_depth": 35, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 300, + "recall_at_k": 0.9796666666666666, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 400324.7733333333, + "depth": 37, + "doc_recall_at_k": 0.777888888888889, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 37.236666666666665, + "metric": "cosine", + "ndcg_at_k": 0.6266916263886889, + "p10_recall_at_k": 1.0, + "p90_depth": 57, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 300, + "recall_at_k": 0.9926666666666665, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 416448.0, + "depth": 20, + "doc_recall_at_k": 0.7745555555555556, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.6258240730658379, + "p10_recall_at_k": 0.9, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.9646666666666666, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9966666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 462528.0, + "depth": 50, + "doc_recall_at_k": 0.777888888888889, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.6267937784557772, + "p10_recall_at_k": 1.0, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.999, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 539328.0, + "depth": 100, + "doc_recall_at_k": 0.777888888888889, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.6267937784557772, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.9996666666666666, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 409592.32, + "depth": 16, + "doc_recall_at_k": 0.7723333333333334, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 15.536666666666667, + "metric": "cosine", + "ndcg_at_k": 0.6251971737176882, + "p10_recall_at_k": 0.8, + "p90_depth": 20, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 300, + "recall_at_k": 0.9306666666666675, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9966666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 422597.12, + "depth": 24, + "doc_recall_at_k": 0.7745555555555556, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 24.003333333333334, + "metric": "cosine", + "ndcg_at_k": 0.6257903451368972, + "p10_recall_at_k": 0.9, + "p90_depth": 35, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 300, + "recall_at_k": 0.9846666666666666, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 442923.52, + "depth": 37, + "doc_recall_at_k": 0.777888888888889, + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 37.236666666666665, + "metric": "cosine", + "ndcg_at_k": 0.6267937784557772, + "p10_recall_at_k": 1.0, + "p90_depth": 57, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 300, + "recall_at_k": 0.9976666666666665, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 192864.0, + "depth": 0, + "doc_recall_at_k": 0.7068333333333333, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 300, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.5501427919742216, + "p10_recall_at_k": 0.4, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 300, + "recall_at_k": 0.6363333333333332, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.5866666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 200704.0, + "depth": 20, + "doc_recall_at_k": 0.7556666666666667, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 300, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.6150679548538306, + "p10_recall_at_k": 0.6, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.8063333333333339, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9766666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 212464.0, + "depth": 50, + "doc_recall_at_k": 0.767888888888889, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 300, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.6237653633002108, + "p10_recall_at_k": 0.8, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.9340000000000004, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9933333333333333, + "vectors": 8036 + }, + { + "bytes_read_per_query": 232064.0, + "depth": 100, + "doc_recall_at_k": 0.7745555555555556, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 300, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.6260214232766527, + "p10_recall_at_k": 0.9, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.974666666666667, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 204242.45333333334, + "depth": 29, + "doc_recall_at_k": 0.7701111111111111, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 300, + "mean_depth": 29.026666666666667, + "metric": "cosine", + "ndcg_at_k": 0.6244001398240199, + "p10_recall_at_k": 0.7, + "p90_depth": 46, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 300, + "recall_at_k": 0.8763333333333352, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.99, + "vectors": 8036 + }, + { + "bytes_read_per_query": 224289.33333333334, + "depth": 80, + "doc_recall_at_k": 0.7712222222222223, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 300, + "mean_depth": 80.16666666666667, + "metric": "cosine", + "ndcg_at_k": 0.6246792340257862, + "p10_recall_at_k": 0.9, + "p90_depth": 159, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 300, + "recall_at_k": 0.9743333333333334, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 270890.29333333333, + "depth": 199, + "doc_recall_at_k": 0.7712222222222223, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 300, + "mean_depth": 199.04666666666665, + "metric": "cosine", + "ndcg_at_k": 0.624528965688292, + "p10_recall_at_k": 1.0, + "p90_depth": 425, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 300, + "recall_at_k": 0.9913333333333333, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 223584.0, + "depth": 20, + "doc_recall_at_k": 0.7556666666666667, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 300, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.6161097014141002, + "p10_recall_at_k": 0.6, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.8063333333333339, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9766666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 269664.0, + "depth": 50, + "doc_recall_at_k": 0.767888888888889, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 300, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.6240728697394486, + "p10_recall_at_k": 0.8, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.9370000000000005, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9933333333333333, + "vectors": 8036 + }, + { + "bytes_read_per_query": 346464.0, + "depth": 100, + "doc_recall_at_k": 0.7745555555555556, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 300, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.6261235753437411, + "p10_recall_at_k": 0.9, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.9786666666666669, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 237448.96, + "depth": 29, + "doc_recall_at_k": 0.7701111111111111, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 300, + "mean_depth": 29.026666666666667, + "metric": "cosine", + "ndcg_at_k": 0.6247076462632577, + "p10_recall_at_k": 0.7, + "p90_depth": 46, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 300, + "recall_at_k": 0.8766666666666686, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.99, + "vectors": 8036 + }, + { + "bytes_read_per_query": 316000.0, + "depth": 80, + "doc_recall_at_k": 0.7712222222222223, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 300, + "mean_depth": 80.16666666666667, + "metric": "cosine", + "ndcg_at_k": 0.6247813860928745, + "p10_recall_at_k": 0.9, + "p90_depth": 159, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 300, + "recall_at_k": 0.9783333333333335, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 498599.68, + "depth": 199, + "doc_recall_at_k": 0.7712222222222223, + "first": "pq24", + "first_bytes": 24, + "k": 10, + "labelled_queries": 300, + "mean_depth": 199.04666666666665, + "metric": "cosine", + "ndcg_at_k": 0.6246311177553804, + "p10_recall_at_k": 1.0, + "p90_depth": 425, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 300, + "recall_at_k": 0.9963333333333333, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 192864.0, + "depth": 0, + "doc_recall_at_k": 0.7296666666666666, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 300, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.5651227816647438, + "p10_recall_at_k": 0.4, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 300, + "recall_at_k": 0.6436666666666667, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.6466666666666666, + "vectors": 8036 + }, + { + "bytes_read_per_query": 200704.0, + "depth": 20, + "doc_recall_at_k": 0.7682222222222221, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 300, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.621897146482731, + "p10_recall_at_k": 0.6, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.819333333333334, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.98, + "vectors": 8036 + }, + { + "bytes_read_per_query": 212464.0, + "depth": 50, + "doc_recall_at_k": 0.772888888888889, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 300, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.6251286121669999, + "p10_recall_at_k": 0.8, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.9460000000000001, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9933333333333333, + "vectors": 8036 + }, + { + "bytes_read_per_query": 232064.0, + "depth": 100, + "doc_recall_at_k": 0.7745555555555556, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 300, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.6256881930698088, + "p10_recall_at_k": 0.9, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.9783333333333334, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9966666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 202511.12, + "depth": 25, + "doc_recall_at_k": 0.7648888888888888, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 300, + "mean_depth": 24.61, + "metric": "cosine", + "ndcg_at_k": 0.6212379845846047, + "p10_recall_at_k": 0.7, + "p90_depth": 35, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 300, + "recall_at_k": 0.8643333333333353, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9866666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 215854.8, + "depth": 59, + "doc_recall_at_k": 0.777888888888889, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 300, + "mean_depth": 58.65, + "metric": "cosine", + "ndcg_at_k": 0.62697082079885, + "p10_recall_at_k": 0.9, + "p90_depth": 95, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 300, + "recall_at_k": 0.9670000000000001, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9933333333333333, + "vectors": 8036 + }, + { + "bytes_read_per_query": 243300.02666666667, + "depth": 129, + "doc_recall_at_k": 0.777888888888889, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 300, + "mean_depth": 128.66333333333333, + "metric": "cosine", + "ndcg_at_k": 0.6266916263886889, + "p10_recall_at_k": 1.0, + "p90_depth": 222, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 300, + "recall_at_k": 0.9896666666666664, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 223584.0, + "depth": 20, + "doc_recall_at_k": 0.7715555555555554, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 300, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.6227756969334982, + "p10_recall_at_k": 0.6, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.819333333333334, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.98, + "vectors": 8036 + }, + { + "bytes_read_per_query": 269664.0, + "depth": 50, + "doc_recall_at_k": 0.772888888888889, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 300, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.6252307642340884, + "p10_recall_at_k": 0.8, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.9493333333333334, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9933333333333333, + "vectors": 8036 + }, + { + "bytes_read_per_query": 346464.0, + "depth": 100, + "doc_recall_at_k": 0.7745555555555556, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 300, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.6257903451368972, + "p10_recall_at_k": 0.9, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.9829999999999999, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9966666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 230664.96, + "depth": 25, + "doc_recall_at_k": 0.7682222222222221, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 300, + "mean_depth": 24.61, + "metric": "cosine", + "ndcg_at_k": 0.6220726079329973, + "p10_recall_at_k": 0.7, + "p90_depth": 35, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 300, + "recall_at_k": 0.8656666666666685, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9866666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 282950.4, + "depth": 59, + "doc_recall_at_k": 0.777888888888889, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 300, + "mean_depth": 58.65, + "metric": "cosine", + "ndcg_at_k": 0.6270729728659384, + "p10_recall_at_k": 0.9, + "p90_depth": 95, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 300, + "recall_at_k": 0.9706666666666668, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9933333333333333, + "vectors": 8036 + }, + { + "bytes_read_per_query": 390490.88, + "depth": 129, + "doc_recall_at_k": 0.777888888888889, + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "labelled_queries": 300, + "mean_depth": 128.66333333333333, + "metric": "cosine", + "ndcg_at_k": 0.6267937784557772, + "p10_recall_at_k": 1.0, + "p90_depth": 222, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 300, + "recall_at_k": 0.9946666666666664, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 192864.0, + "depth": 0, + "doc_recall_at_k": 0.7118888888888889, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 300, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.5644372035080698, + "p10_recall_at_k": 0.4, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 300, + "recall_at_k": 0.6673333333333331, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.6566666666666666, + "vectors": 8036 + }, + { + "bytes_read_per_query": 200704.0, + "depth": 20, + "doc_recall_at_k": 0.7505555555555555, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 300, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.6142566481879647, + "p10_recall_at_k": 0.6, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.8473333333333344, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.98, + "vectors": 8036 + }, + { + "bytes_read_per_query": 212464.0, + "depth": 50, + "doc_recall_at_k": 0.7645555555555555, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 300, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.621359034986275, + "p10_recall_at_k": 0.9, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.9553333333333336, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9966666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 232064.0, + "depth": 100, + "doc_recall_at_k": 0.7745555555555556, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 300, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.6261022330789139, + "p10_recall_at_k": 0.9, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.9846666666666666, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9966666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 202235.41333333333, + "depth": 24, + "doc_recall_at_k": 0.7553888888888889, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 300, + "mean_depth": 23.906666666666666, + "metric": "cosine", + "ndcg_at_k": 0.6186005990424303, + "p10_recall_at_k": 0.7, + "p90_depth": 36, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 300, + "recall_at_k": 0.8856666666666679, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.99, + "vectors": 8036 + }, + { + "bytes_read_per_query": 214177.04, + "depth": 54, + "doc_recall_at_k": 0.7712222222222223, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 300, + "mean_depth": 54.37, + "metric": "cosine", + "ndcg_at_k": 0.6246666445520025, + "p10_recall_at_k": 0.9, + "p90_depth": 90, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 300, + "recall_at_k": 0.969, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 0.9966666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 237644.77333333332, + "depth": 114, + "doc_recall_at_k": 0.777888888888889, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 300, + "mean_depth": 114.23666666666666, + "metric": "cosine", + "ndcg_at_k": 0.6266916263886889, + "p10_recall_at_k": 1.0, + "p90_depth": 199, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 300, + "recall_at_k": 0.9906666666666665, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 223584.0, + "depth": 20, + "doc_recall_at_k": 0.7505555555555555, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 300, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.614095403038423, + "p10_recall_at_k": 0.6, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.8476666666666677, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.98, + "vectors": 8036 + }, + { + "bytes_read_per_query": 269664.0, + "depth": 50, + "doc_recall_at_k": 0.7645555555555555, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 300, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.621230108913608, + "p10_recall_at_k": 0.9, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.9590000000000002, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9966666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 346464.0, + "depth": 100, + "doc_recall_at_k": 0.7745555555555556, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 300, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.6262043851460022, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.9893333333333332, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9966666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 229584.64, + "depth": 24, + "doc_recall_at_k": 0.7553888888888889, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 300, + "mean_depth": 23.906666666666666, + "metric": "cosine", + "ndcg_at_k": 0.6184716729697632, + "p10_recall_at_k": 0.7, + "p90_depth": 36, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 300, + "recall_at_k": 0.8866666666666682, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.99, + "vectors": 8036 + }, + { + "bytes_read_per_query": 276376.32, + "depth": 54, + "doc_recall_at_k": 0.7712222222222223, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 300, + "mean_depth": 54.37, + "metric": "cosine", + "ndcg_at_k": 0.6247687966190909, + "p10_recall_at_k": 0.9, + "p90_depth": 90, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 300, + "recall_at_k": 0.9733333333333334, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9966666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 368331.52, + "depth": 114, + "doc_recall_at_k": 0.777888888888889, + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "labelled_queries": 300, + "mean_depth": 114.23666666666666, + "metric": "cosine", + "ndcg_at_k": 0.6267937784557772, + "p10_recall_at_k": 1.0, + "p90_depth": 199, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 300, + "recall_at_k": 0.9956666666666665, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 771456.0, + "depth": 0, + "doc_recall_at_k": 0.7863888888888888, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 300, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.6303028117112374, + "p10_recall_at_k": 0.8, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 300, + "recall_at_k": 0.874666666666669, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.89, + "vectors": 8036 + }, + { + "bytes_read_per_query": 779296.0, + "depth": 20, + "doc_recall_at_k": 0.777888888888889, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 300, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.6266916263886889, + "p10_recall_at_k": 0.9, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.9880000000000001, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 791056.0, + "depth": 50, + "doc_recall_at_k": 0.777888888888889, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 300, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.6266916263886889, + "p10_recall_at_k": 1.0, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.9946666666666666, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 810656.0, + "depth": 100, + "doc_recall_at_k": 0.777888888888889, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 300, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.6266916263886889, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.995, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 776566.3733333333, + "depth": 13, + "doc_recall_at_k": 0.7788888888888889, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 300, + "mean_depth": 13.036666666666667, + "metric": "cosine", + "ndcg_at_k": 0.6270461034278793, + "p10_recall_at_k": 0.9, + "p90_depth": 16, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 300, + "recall_at_k": 0.9573333333333338, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 778170.96, + "depth": 17, + "doc_recall_at_k": 0.777888888888889, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 300, + "mean_depth": 17.13, + "metric": "cosine", + "ndcg_at_k": 0.6266916263886889, + "p10_recall_at_k": 0.9, + "p90_depth": 24, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 300, + "recall_at_k": 0.9866666666666667, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 780242.0266666666, + "depth": 22, + "doc_recall_at_k": 0.777888888888889, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 300, + "mean_depth": 22.413333333333334, + "metric": "cosine", + "ndcg_at_k": 0.6266916263886889, + "p10_recall_at_k": 1.0, + "p90_depth": 33, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 300, + "recall_at_k": 0.9943333333333334, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 802176.0, + "depth": 20, + "doc_recall_at_k": 0.777888888888889, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 300, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.6267937784557772, + "p10_recall_at_k": 1.0, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.9930000000000001, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 848256.0, + "depth": 50, + "doc_recall_at_k": 0.777888888888889, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 300, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.6267937784557772, + "p10_recall_at_k": 1.0, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.9996666666666666, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 925056.0, + "depth": 100, + "doc_recall_at_k": 0.777888888888889, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 300, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.6267937784557772, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 1.0, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 791480.32, + "depth": 13, + "doc_recall_at_k": 0.7822222222222222, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 300, + "mean_depth": 13.036666666666667, + "metric": "cosine", + "ndcg_at_k": 0.6281118049160272, + "p10_recall_at_k": 0.9, + "p90_depth": 16, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 300, + "recall_at_k": 0.9593333333333339, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 797767.68, + "depth": 17, + "doc_recall_at_k": 0.777888888888889, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 300, + "mean_depth": 17.13, + "metric": "cosine", + "ndcg_at_k": 0.6267937784557772, + "p10_recall_at_k": 1.0, + "p90_depth": 24, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 300, + "recall_at_k": 0.9906666666666666, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 805882.88, + "depth": 22, + "doc_recall_at_k": 0.777888888888889, + "first": "pq96", + "first_bytes": 96, + "k": 10, + "labelled_queries": 300, + "mean_depth": 22.413333333333334, + "metric": "cosine", + "ndcg_at_k": 0.6267937784557772, + "p10_recall_at_k": 1.0, + "p90_depth": 33, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 300, + "recall_at_k": 0.9990000000000001, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 771456.0, + "depth": 0, + "doc_recall_at_k": 0.7661111111111112, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 300, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.6216832175000905, + "p10_recall_at_k": 0.8, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 300, + "recall_at_k": 0.9156666666666687, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.9166666666666666, + "vectors": 8036 + }, + { + "bytes_read_per_query": 779296.0, + "depth": 20, + "doc_recall_at_k": 0.777888888888889, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 300, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.6266916263886889, + "p10_recall_at_k": 1.0, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.9936666666666667, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 791056.0, + "depth": 50, + "doc_recall_at_k": 0.777888888888889, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 300, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.6266916263886889, + "p10_recall_at_k": 1.0, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.995, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 810656.0, + "depth": 100, + "doc_recall_at_k": 0.777888888888889, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 300, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.6266916263886889, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.995, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 776127.3333333334, + "depth": 12, + "doc_recall_at_k": 0.777888888888889, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 300, + "mean_depth": 11.916666666666666, + "metric": "cosine", + "ndcg_at_k": 0.6266916263886889, + "p10_recall_at_k": 0.9, + "p90_depth": 14, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 300, + "recall_at_k": 0.9693333333333337, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 777113.8666666667, + "depth": 14, + "doc_recall_at_k": 0.7812222222222223, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 300, + "mean_depth": 14.433333333333334, + "metric": "cosine", + "ndcg_at_k": 0.6276551758097485, + "p10_recall_at_k": 0.9, + "p90_depth": 18, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 300, + "recall_at_k": 0.9886666666666667, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 778210.16, + "depth": 17, + "doc_recall_at_k": 0.777888888888889, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 300, + "mean_depth": 17.23, + "metric": "cosine", + "ndcg_at_k": 0.6266916263886889, + "p10_recall_at_k": 1.0, + "p90_depth": 23, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 300, + "recall_at_k": 0.9933333333333333, + "rescore_ms": 0, + "second": "int8", + "second_bytes": 392, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 802176.0, + "depth": 20, + "doc_recall_at_k": 0.777888888888889, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 300, + "mean_depth": 20.0, + "metric": "cosine", + "ndcg_at_k": 0.6267937784557772, + "p10_recall_at_k": 1.0, + "p90_depth": 20, + "policy": { + "depth": 20, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.9986666666666667, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 848256.0, + "depth": 50, + "doc_recall_at_k": 0.777888888888889, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 300, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.6267937784557772, + "p10_recall_at_k": 1.0, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 1.0, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 925056.0, + "depth": 100, + "doc_recall_at_k": 0.777888888888889, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 300, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.6267937784557772, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 1.0, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 789760.0, + "depth": 12, + "doc_recall_at_k": 0.777888888888889, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 300, + "mean_depth": 11.916666666666666, + "metric": "cosine", + "ndcg_at_k": 0.6267937784557772, + "p10_recall_at_k": 0.9, + "p90_depth": 14, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 300, + "recall_at_k": 0.9723333333333335, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 793625.6, + "depth": 14, + "doc_recall_at_k": 0.7812222222222223, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 300, + "mean_depth": 14.433333333333334, + "metric": "cosine", + "ndcg_at_k": 0.6277573278768368, + "p10_recall_at_k": 1.0, + "p90_depth": 18, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 300, + "recall_at_k": 0.9933333333333335, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 797921.28, + "depth": 17, + "doc_recall_at_k": 0.777888888888889, + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "labelled_queries": 300, + "mean_depth": 17.23, + "metric": "cosine", + "ndcg_at_k": 0.6267937784557772, + "p10_recall_at_k": 1.0, + "p90_depth": 23, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 300, + "recall_at_k": 0.9983333333333333, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + } + ], + "profiles": [ + { + "depth_for_recall": [ + [ + 0.9, + 24 + ], + [ + 0.95, + 36 + ], + [ + 0.99, + 85 + ], + [ + 1.0, + 432 + ] + ], + "first": "binary", + "first_bytes": 48, + "k": 10, + "median_full_depth": 26.0, + "metric": "cosine", + "neighbour_band": 0.03073228895664215, + "queries": 300, + "rank_noise": 0.027713695127966537, + "recall_at_k": 0.7049999999999998, + "vectors": 8036 + }, + { + "depth_for_recall": [ + [ + 0.9, + 21 + ], + [ + 0.95, + 31 + ], + [ + 0.99, + 60 + ], + [ + 1.0, + 372 + ] + ], + "first": "binary-centred", + "first_bytes": 48, + "k": 10, + "median_full_depth": 23.0, + "metric": "cosine", + "neighbour_band": 0.03073228895664215, + "queries": 300, + "rank_noise": 0.026426586664381465, + "recall_at_k": 0.718666666666667, + "vectors": 8036 + }, + { + "depth_for_recall": [ + [ + 0.9, + 17 + ], + [ + 0.95, + 22 + ], + [ + 0.99, + 45 + ], + [ + 1.0, + 261 + ] + ], + "first": "pq48", + "first_bytes": 48, + "k": 10, + "median_full_depth": 18.0, + "metric": "cosine", + "neighbour_band": 0.03073228895664215, + "queries": 300, + "rank_noise": 0.022971436105667608, + "recall_at_k": 0.7640000000000003, + "vectors": 8036 + }, + { + "depth_for_recall": [ + [ + 0.9, + 20 + ], + [ + 0.95, + 31 + ], + [ + 0.99, + 57 + ], + [ + 1.0, + 259 + ] + ], + "first": "mixed48", + "first_bytes": 48, + "k": 10, + "median_full_depth": 23.0, + "metric": "cosine", + "neighbour_band": 0.03073228895664215, + "queries": 300, + "rank_noise": 0.026419433216495397, + "recall_at_k": 0.7176666666666672, + "vectors": 8036 + }, + { + "depth_for_recall": [ + [ + 0.9, + 16 + ], + [ + 0.95, + 21 + ], + [ + 0.99, + 37 + ], + [ + 1.0, + 113 + ] + ], + "first": "mixed48-pca", + "first_bytes": 48, + "k": 10, + "median_full_depth": 17.0, + "metric": "cosine", + "neighbour_band": 0.03073228895664215, + "queries": 300, + "rank_noise": 0.021233104542443853, + "recall_at_k": 0.7753333333333347, + "vectors": 8036 + }, + { + "depth_for_recall": [ + [ + 0.9, + 14 + ], + [ + 0.95, + 18 + ], + [ + 0.99, + 31 + ], + [ + 1.0, + 106 + ] + ], + "first": "mixed48-pca-w", + "first_bytes": 48, + "k": 10, + "median_full_depth": 16.0, + "metric": "cosine", + "neighbour_band": 0.03073228895664215, + "queries": 300, + "rank_noise": 0.01922714183692469, + "recall_at_k": 0.8043333333333348, + "vectors": 8036 + }, + { + "depth_for_recall": [ + [ + 0.9, + 36 + ], + [ + 0.95, + 58 + ], + [ + 0.99, + 168 + ], + [ + 1.0, + 1083 + ] + ], + "first": "pq24", + "first_bytes": 24, + "k": 10, + "median_full_depth": 37.0, + "metric": "cosine", + "neighbour_band": 0.03073228895664215, + "queries": 300, + "rank_noise": 0.03588049972806333, + "recall_at_k": 0.6363333333333332, + "vectors": 8036 + }, + { + "depth_for_recall": [ + [ + 0.9, + 32 + ], + [ + 0.95, + 51 + ], + [ + 0.99, + 136 + ], + [ + 1.0, + 372 + ] + ], + "first": "mixed24-pca", + "first_bytes": 24, + "k": 10, + "median_full_depth": 38.0, + "metric": "cosine", + "neighbour_band": 0.03073228895664215, + "queries": 300, + "rank_noise": 0.033692169717293906, + "recall_at_k": 0.6436666666666667, + "vectors": 8036 + }, + { + "depth_for_recall": [ + [ + 0.9, + 29 + ], + [ + 0.95, + 45 + ], + [ + 0.99, + 106 + ], + [ + 1.0, + 761 + ] + ], + "first": "mixed24-pca-w", + "first_bytes": 24, + "k": 10, + "median_full_depth": 32.0, + "metric": "cosine", + "neighbour_band": 0.03073228895664215, + "queries": 300, + "rank_noise": 0.033366440819880216, + "recall_at_k": 0.6673333333333331, + "vectors": 8036 + }, + { + "depth_for_recall": [ + [ + 0.9, + 11 + ], + [ + 0.95, + 13 + ], + [ + 0.99, + 19 + ], + [ + 1.0, + 60 + ] + ], + "first": "pq96", + "first_bytes": 96, + "k": 10, + "median_full_depth": 12.0, + "metric": "cosine", + "neighbour_band": 0.03073228895664215, + "queries": 300, + "rank_noise": 0.011711888456250843, + "recall_at_k": 0.874666666666669, + "vectors": 8036 + }, + { + "depth_for_recall": [ + [ + 0.9, + 10 + ], + [ + 0.95, + 11 + ], + [ + 0.99, + 15 + ], + [ + 1.0, + 27 + ] + ], + "first": "mixed96-pca-w", + "first_bytes": 96, + "k": 10, + "median_full_depth": 11.0, + "metric": "cosine", + "neighbour_band": 0.03073228895664215, + "queries": 300, + "rank_noise": 0.008109765457907094, + "recall_at_k": 0.9156666666666687, + "vectors": 8036 + } + ] +} diff --git a/experiments/2026-11-hybrid-adaptive-matrix/results/summary.txt b/experiments/2026-11-hybrid-adaptive-matrix/results/summary.txt new file mode 100644 index 0000000..49ace71 --- /dev/null +++ b/experiments/2026-11-hybrid-adaptive-matrix/results/summary.txt @@ -0,0 +1,240 @@ +10 collections: arguana-bge-small, arguana-minilm, cosqa-bge-small, cosqa-minilm, fiqa-bge-small, fiqa-minilm, nfcorpus-bge-small, nfcorpus-minilm, scifact-bge-small, scifact-minilm + +1. The matrix (means over collections). ram = scanned bytes/vector, disk = second-stage bytes/vector, + read = bytes touched per query as a fraction of a full f32 scan +scan codec second policy ram disk depth p90 recall@10 p10 top1 ndcg loss read +binary none none 48 0 0.0 0 0.6685 0.410 0.707 0.0240 0.0312 +binary f32 fixed 100 48 1536 100.0 100 0.9808 0.950 0.998 -0.0001 0.0400 +binary f32 fixed 20 48 1536 20.0 20 0.8405 0.620 0.978 0.0035 0.0330 +binary f32 fixed 50 48 1536 50.0 50 0.9482 0.860 0.994 0.0003 0.0356 +binary f32 margin 1σ 48 1536 23.2 35 0.8689 0.690 0.986 0.0022 0.0332 +binary f32 margin 2σ 48 1536 51.9 88 0.9627 0.880 0.994 -0.0006 0.0356 +binary f32 margin 3σ 48 1536 114.8 212 0.9911 0.980 0.999 -0.0001 0.0407 +binary int8 fixed 100 48 392 100.0 100 0.9739 0.920 0.993 -0.0001 0.0335 +binary int8 fixed 20 48 392 20.0 20 0.8392 0.620 0.974 0.0033 0.0317 +binary int8 fixed 50 48 392 50.0 50 0.9439 0.840 0.989 0.0004 0.0324 +binary int8 margin 1σ 48 392 23.2 35 0.8675 0.690 0.982 0.0021 0.0317 +binary int8 margin 2σ 48 392 51.9 88 0.9576 0.880 0.989 -0.0006 0.0324 +binary int8 margin 3σ 48 392 114.8 212 0.9831 0.950 0.993 -0.0000 0.0337 +binary-centred none none 48 0 0.0 0 0.6977 0.430 0.724 0.0236 0.0312 +binary-centred f32 fixed 100 48 1536 100.0 100 0.9910 0.980 1.000 0.0000 0.0400 +binary-centred f32 fixed 20 48 1536 20.0 20 0.8691 0.660 0.983 0.0026 0.0330 +binary-centred f32 fixed 50 48 1536 50.0 50 0.9662 0.900 0.997 0.0002 0.0356 +binary-centred f32 margin 1σ 48 1536 25.7 40 0.9112 0.800 0.995 0.0009 0.0334 +binary-centred f32 margin 2σ 48 1536 65.0 117 0.9840 0.940 0.999 0.0000 0.0363 +binary-centred f32 margin 3σ 48 1536 152.2 304 0.9979 1.000 1.000 0.0001 0.0430 +binary-centred int8 fixed 100 48 392 100.0 100 0.9830 0.960 0.994 0.0000 0.0335 +binary-centred int8 fixed 20 48 392 20.0 20 0.8672 0.660 0.978 0.0024 0.0317 +binary-centred int8 fixed 50 48 392 50.0 50 0.9605 0.880 0.991 0.0002 0.0324 +binary-centred int8 margin 1σ 48 392 25.7 40 0.9084 0.790 0.990 0.0006 0.0318 +binary-centred int8 margin 2σ 48 392 65.0 117 0.9766 0.920 0.994 0.0001 0.0326 +binary-centred int8 margin 3σ 48 392 152.2 304 0.9888 0.970 0.994 0.0000 0.0343 +mixed24-pca none none 24 0 0.0 0 0.6314 0.370 0.647 0.0497 0.0156 +mixed24-pca f32 fixed 100 24 1536 100.0 100 0.9767 0.920 0.997 0.0014 0.0243 +mixed24-pca f32 fixed 20 24 1536 20.0 20 0.8070 0.540 0.960 0.0106 0.0174 +mixed24-pca f32 fixed 50 24 1536 50.0 50 0.9372 0.810 0.991 0.0037 0.0200 +mixed24-pca f32 margin 1σ 24 1536 30.8 49 0.8804 0.730 0.985 0.0062 0.0181 +mixed24-pca f32 margin 2σ 24 1536 88.8 160 0.9711 0.910 0.998 0.0014 0.0221 +mixed24-pca f32 margin 3σ 24 1536 212.2 376 0.9941 0.990 1.000 0.0001 0.0308 +mixed24-pca int8 fixed 100 24 392 100.0 100 0.9700 0.900 0.992 0.0014 0.0178 +mixed24-pca int8 fixed 20 24 392 20.0 20 0.8062 0.540 0.956 0.0108 0.0161 +mixed24-pca int8 fixed 50 24 392 50.0 50 0.9330 0.810 0.986 0.0039 0.0167 +mixed24-pca int8 margin 1σ 24 392 30.8 49 0.8780 0.730 0.981 0.0064 0.0162 +mixed24-pca int8 margin 2σ 24 392 88.8 160 0.9648 0.900 0.992 0.0015 0.0173 +mixed24-pca int8 margin 3σ 24 392 212.2 376 0.9853 0.960 0.994 0.0002 0.0195 +mixed24-pca-w none none 24 0 0.0 0 0.6347 0.370 0.647 0.0523 0.0156 +mixed24-pca-w f32 fixed 100 24 1536 100.0 100 0.9772 0.930 0.997 0.0018 0.0243 +mixed24-pca-w f32 fixed 20 24 1536 20.0 20 0.8154 0.560 0.962 0.0126 0.0174 +mixed24-pca-w f32 fixed 50 24 1536 50.0 50 0.9373 0.830 0.992 0.0047 0.0200 +mixed24-pca-w f32 margin 1σ 24 1536 33.2 55 0.8883 0.750 0.986 0.0076 0.0182 +mixed24-pca-w f32 margin 2σ 24 1536 99.7 187 0.9726 0.910 0.997 0.0015 0.0227 +mixed24-pca-w f32 margin 3σ 24 1536 227.6 395 0.9935 0.980 0.999 0.0003 0.0315 +mixed24-pca-w int8 fixed 100 24 392 100.0 100 0.9702 0.900 0.992 0.0018 0.0178 +mixed24-pca-w int8 fixed 20 24 392 20.0 20 0.8144 0.560 0.958 0.0123 0.0161 +mixed24-pca-w int8 fixed 50 24 392 50.0 50 0.9329 0.820 0.987 0.0048 0.0167 +mixed24-pca-w int8 margin 1σ 24 392 33.2 55 0.8860 0.740 0.981 0.0077 0.0163 +mixed24-pca-w int8 margin 2σ 24 392 99.7 187 0.9656 0.900 0.992 0.0016 0.0174 +mixed24-pca-w int8 margin 3σ 24 392 227.6 395 0.9847 0.960 0.994 0.0003 0.0197 +mixed48 none none 48 0 0.0 0 0.6959 0.430 0.721 0.0242 0.0312 +mixed48 f32 fixed 100 48 1536 100.0 100 0.9909 0.980 0.999 -0.0001 0.0400 +mixed48 f32 fixed 20 48 1536 20.0 20 0.8679 0.660 0.983 0.0026 0.0330 +mixed48 f32 fixed 50 48 1536 50.0 50 0.9657 0.900 0.997 -0.0000 0.0356 +mixed48 f32 margin 1σ 48 1536 25.6 40 0.9090 0.790 0.994 0.0008 0.0333 +mixed48 f32 margin 2σ 48 1536 64.6 118 0.9831 0.940 0.999 0.0001 0.0363 +mixed48 f32 margin 3σ 48 1536 150.2 300 0.9977 1.000 1.000 -0.0001 0.0429 +mixed48 int8 fixed 100 48 392 100.0 100 0.9830 0.970 0.993 -0.0001 0.0335 +mixed48 int8 fixed 20 48 392 20.0 20 0.8662 0.660 0.977 0.0024 0.0317 +mixed48 int8 fixed 50 48 392 50.0 50 0.9599 0.880 0.991 -0.0000 0.0324 +mixed48 int8 margin 1σ 48 392 25.6 40 0.9062 0.780 0.989 0.0007 0.0318 +mixed48 int8 margin 2σ 48 392 64.6 118 0.9760 0.920 0.993 0.0001 0.0325 +mixed48 int8 margin 3σ 48 392 150.2 300 0.9886 0.970 0.994 -0.0001 0.0342 +mixed48-pca none none 48 0 0.0 0 0.7549 0.540 0.770 0.0167 0.0312 +mixed48-pca f32 fixed 100 48 1536 100.0 100 0.9982 1.000 1.000 -0.0001 0.0400 +mixed48-pca f32 fixed 20 48 1536 20.0 20 0.9204 0.770 0.995 0.0015 0.0330 +mixed48-pca f32 fixed 50 48 1536 50.0 50 0.9885 0.970 0.999 -0.0000 0.0356 +mixed48-pca f32 margin 1σ 48 1536 19.1 28 0.9195 0.810 0.997 0.0012 0.0328 +mixed48-pca f32 margin 2σ 48 1536 34.9 57 0.9822 0.930 1.000 0.0001 0.0341 +mixed48-pca f32 margin 3σ 48 1536 64.2 116 0.9967 0.990 1.000 -0.0000 0.0363 +mixed48-pca int8 fixed 100 48 392 100.0 100 0.9890 0.970 0.994 -0.0000 0.0335 +mixed48-pca int8 fixed 20 48 392 20.0 20 0.9182 0.770 0.990 0.0012 0.0317 +mixed48-pca int8 fixed 50 48 392 50.0 50 0.9809 0.950 0.994 0.0000 0.0324 +mixed48-pca int8 margin 1σ 48 392 19.1 28 0.9170 0.800 0.991 0.0011 0.0317 +mixed48-pca int8 margin 2σ 48 392 34.9 57 0.9752 0.920 0.994 0.0001 0.0320 +mixed48-pca int8 margin 3σ 48 392 64.2 116 0.9877 0.960 0.994 -0.0000 0.0325 +mixed48-pca-w none none 48 0 0.0 0 0.7678 0.550 0.776 0.0177 0.0312 +mixed48-pca-w f32 fixed 100 48 1536 100.0 100 0.9985 1.000 1.000 0.0001 0.0400 +mixed48-pca-w f32 fixed 20 48 1536 20.0 20 0.9309 0.800 0.994 0.0011 0.0330 +mixed48-pca-w f32 fixed 50 48 1536 50.0 50 0.9911 0.980 0.999 0.0000 0.0356 +mixed48-pca-w f32 margin 1σ 48 1536 19.4 28 0.9295 0.830 0.997 0.0018 0.0328 +mixed48-pca-w f32 margin 2σ 48 1536 36.0 60 0.9832 0.930 1.000 0.0006 0.0341 +mixed48-pca-w f32 margin 3σ 48 1536 67.8 123 0.9961 0.980 1.000 0.0000 0.0363 +mixed48-pca-w int8 fixed 100 48 392 100.0 100 0.9895 0.970 0.994 0.0002 0.0335 +mixed48-pca-w int8 fixed 20 48 392 20.0 20 0.9282 0.800 0.989 0.0012 0.0317 +mixed48-pca-w int8 fixed 50 48 392 50.0 50 0.9833 0.970 0.994 0.0000 0.0324 +mixed48-pca-w int8 margin 1σ 48 392 19.4 28 0.9265 0.820 0.992 0.0020 0.0317 +mixed48-pca-w int8 margin 2σ 48 392 36.0 60 0.9759 0.920 0.994 0.0006 0.0320 +mixed48-pca-w int8 margin 3σ 48 392 67.8 123 0.9872 0.960 0.994 0.0001 0.0325 +mixed96-pca-w none none 96 0 0.0 0 0.8855 0.740 0.900 0.0022 0.0625 +mixed96-pca-w f32 fixed 100 96 1536 100.0 100 1.0000 1.000 1.000 0.0000 0.0712 +mixed96-pca-w f32 fixed 20 96 1536 20.0 20 0.9901 0.970 0.999 0.0002 0.0642 +mixed96-pca-w f32 fixed 50 96 1536 50.0 50 0.9999 1.000 1.000 0.0000 0.0669 +mixed96-pca-w f32 margin 1σ 96 1536 13.6 18 0.9633 0.910 1.000 0.0001 0.0636 +mixed96-pca-w f32 margin 2σ 96 1536 17.6 25 0.9913 0.980 1.000 -0.0001 0.0640 +mixed96-pca-w f32 margin 3σ 96 1536 22.9 35 0.9980 1.000 1.000 -0.0001 0.0644 +mixed96-pca-w int8 fixed 100 96 392 100.0 100 0.9905 0.970 0.994 0.0001 0.0647 +mixed96-pca-w int8 fixed 20 96 392 20.0 20 0.9832 0.960 0.994 0.0001 0.0629 +mixed96-pca-w int8 fixed 50 96 392 50.0 50 0.9904 0.970 0.994 0.0001 0.0636 +mixed96-pca-w int8 margin 1σ 96 392 13.6 18 0.9592 0.900 0.994 0.0001 0.0628 +mixed96-pca-w int8 margin 2σ 96 392 17.6 25 0.9837 0.940 0.994 -0.0001 0.0629 +mixed96-pca-w int8 margin 3σ 96 392 22.9 35 0.9890 0.960 0.994 0.0000 0.0630 +pq24 none none 24 0 0.0 0 0.6058 0.330 0.614 0.0586 0.0156 +pq24 f32 fixed 100 24 1536 100.0 100 0.9680 0.890 0.995 0.0020 0.0243 +pq24 f32 fixed 20 24 1536 20.0 20 0.7791 0.500 0.943 0.0119 0.0174 +pq24 f32 fixed 50 24 1536 50.0 50 0.9182 0.780 0.983 0.0046 0.0200 +pq24 f32 margin 1σ 24 1536 42.1 74 0.8870 0.750 0.986 0.0051 0.0188 +pq24 f32 margin 2σ 24 1536 144.0 295 0.9792 0.920 0.999 0.0005 0.0256 +pq24 f32 margin 3σ 24 1536 335.5 626 0.9971 1.000 1.000 0.0002 0.0395 +pq24 int8 fixed 100 24 392 100.0 100 0.9620 0.880 0.989 0.0020 0.0178 +pq24 int8 fixed 20 24 392 20.0 20 0.7781 0.500 0.939 0.0119 0.0161 +pq24 int8 fixed 50 24 392 50.0 50 0.9146 0.780 0.978 0.0048 0.0167 +pq24 int8 margin 1σ 24 392 42.1 74 0.8844 0.740 0.981 0.0052 0.0164 +pq24 int8 margin 2σ 24 392 144.0 295 0.9720 0.910 0.993 0.0006 0.0182 +pq24 int8 margin 3σ 24 392 335.5 626 0.9881 0.960 0.994 0.0003 0.0217 +pq48 none none 48 0 0.0 0 0.7467 0.520 0.770 0.0160 0.0312 +pq48 f32 fixed 100 48 1536 100.0 100 0.9977 1.000 1.000 -0.0001 0.0400 +pq48 f32 fixed 20 48 1536 20.0 20 0.9125 0.750 0.989 0.0019 0.0330 +pq48 f32 fixed 50 48 1536 50.0 50 0.9863 0.960 0.999 0.0002 0.0356 +pq48 f32 margin 1σ 48 1536 20.8 32 0.9240 0.810 0.995 0.0011 0.0329 +pq48 f32 margin 2σ 48 1536 40.9 71 0.9852 0.940 0.999 0.0002 0.0345 +pq48 f32 margin 3σ 48 1536 80.4 152 0.9979 1.000 1.000 -0.0001 0.0374 +pq48 int8 fixed 100 48 392 100.0 100 0.9888 0.970 0.994 -0.0000 0.0335 +pq48 int8 fixed 20 48 392 20.0 20 0.9098 0.750 0.984 0.0019 0.0317 +pq48 int8 fixed 50 48 392 50.0 50 0.9788 0.930 0.993 0.0002 0.0324 +pq48 int8 margin 1σ 48 392 20.8 32 0.9210 0.810 0.990 0.0010 0.0317 +pq48 int8 margin 2σ 48 392 40.9 71 0.9777 0.930 0.994 0.0002 0.0321 +pq48 int8 margin 3σ 48 392 80.4 152 0.9887 0.970 0.994 -0.0000 0.0328 +pq96 none none 96 0 0.0 0 0.8704 0.730 0.879 0.0031 0.0625 +pq96 f32 fixed 100 96 1536 100.0 100 0.9999 1.000 1.000 0.0000 0.0712 +pq96 f32 fixed 20 96 1536 20.0 20 0.9864 0.950 0.999 -0.0001 0.0642 +pq96 f32 fixed 50 96 1536 50.0 50 0.9997 1.000 1.000 -0.0001 0.0669 +pq96 f32 margin 1σ 96 1536 13.9 18 0.9580 0.880 0.998 0.0000 0.0637 +pq96 f32 margin 2σ 96 1536 18.3 26 0.9898 0.960 0.999 -0.0001 0.0640 +pq96 f32 margin 3σ 96 1536 24.2 37 0.9982 1.000 1.000 -0.0000 0.0645 +pq96 int8 fixed 100 96 392 100.0 100 0.9905 0.970 0.994 0.0001 0.0647 +pq96 int8 fixed 20 96 392 20.0 20 0.9791 0.910 0.994 -0.0002 0.0629 +pq96 int8 fixed 50 96 392 50.0 50 0.9903 0.970 0.994 -0.0000 0.0636 +pq96 int8 margin 1σ 96 392 13.9 18 0.9540 0.880 0.992 0.0002 0.0628 +pq96 int8 margin 2σ 96 392 18.3 26 0.9821 0.930 0.994 -0.0001 0.0629 +pq96 int8 margin 3σ 96 392 24.2 37 0.9890 0.970 0.994 -0.0000 0.0630 + +2. Adaptive storage alone (no re-score), by byte budget; per model, then both + 24 B mixed24-pca-w recall@10 0.6347 (bge 0.5924, MiniLM 0.6769) p10 0.370 ndcg loss 0.0523 + 24 B mixed24-pca recall@10 0.6314 (bge 0.5963, MiniLM 0.6665) p10 0.370 ndcg loss 0.0497 + 24 B pq24 recall@10 0.6058 (bge 0.5874, MiniLM 0.6242) p10 0.330 ndcg loss 0.0586 + 48 B mixed48-pca-w recall@10 0.7678 (bge 0.7226, MiniLM 0.8131) p10 0.550 ndcg loss 0.0177 + 48 B mixed48-pca recall@10 0.7549 (bge 0.7167, MiniLM 0.7930) p10 0.540 ndcg loss 0.0167 + 48 B pq48 recall@10 0.7467 (bge 0.7242, MiniLM 0.7693) p10 0.520 ndcg loss 0.0160 + 48 B binary-centred recall@10 0.6977 (bge 0.6673, MiniLM 0.7281) p10 0.430 ndcg loss 0.0236 + 48 B mixed48 recall@10 0.6959 (bge 0.6634, MiniLM 0.7285) p10 0.430 ndcg loss 0.0242 + 48 B binary recall@10 0.6685 (bge 0.6223, MiniLM 0.7147) p10 0.410 ndcg loss 0.0240 + 96 B mixed96-pca-w recall@10 0.8855 (bge 0.8551, MiniLM 0.9159) p10 0.740 ndcg loss 0.0022 + 96 B pq96 recall@10 0.8704 (bge 0.8554, MiniLM 0.8854) p10 0.730 ndcg loss 0.0031 + +3. Adaptive depth against fixed depth at the same mean depth (per collection, then averaged) + scan codec second margin mean depth Δrecall Δp10 Δndcg p90/mean cells + binary f32 1 23.2 +0.0126 +0.033 +0.0007 1.49 10 + binary f32 2 51.9 +0.0145 +0.036 +0.0009 1.69 10 + binary f32 3 53.6 +0.0090 +0.024 -0.0004 1.93 3 + binary int8 1 23.2 +0.0130 +0.037 +0.0005 1.49 10 + binary int8 2 51.9 +0.0142 +0.049 +0.0008 1.69 10 + binary int8 3 53.6 +0.0088 +0.036 -0.0004 1.93 3 + binary-centred f32 1 25.7 +0.0196 +0.065 +0.0006 1.51 10 + binary-centred f32 2 54.1 +0.0125 +0.031 +0.0003 1.71 9 + binary-centred f32 3 57.2 +0.0050 +0.012 +0.0001 1.89 2 + binary-centred int8 1 25.7 +0.0197 +0.060 +0.0007 1.51 10 + binary-centred int8 2 54.1 +0.0116 +0.027 +0.0003 1.71 9 + binary-centred int8 3 57.2 +0.0046 +0.033 +0.0001 1.89 2 + mixed24-pca f32 1 30.8 +0.0156 +0.048 +0.0019 1.54 10 + mixed24-pca f32 2 66.2 +0.0106 +0.055 +0.0005 1.69 8 + mixed24-pca f32 3 67.9 +0.0043 +0.013 +0.0000 1.85 2 + mixed24-pca int8 1 30.8 +0.0157 +0.051 +0.0019 1.54 10 + mixed24-pca int8 2 66.2 +0.0102 +0.042 +0.0004 1.69 8 + mixed24-pca int8 3 67.9 +0.0039 +0.054 -0.0001 1.85 2 + mixed24-pca-w f32 1 33.2 +0.0150 +0.036 +0.0021 1.56 10 + mixed24-pca-w f32 2 53.5 +0.0110 +0.031 +0.0013 1.65 6 + mixed24-pca-w f32 3 76.0 +0.0042 +0.006 +0.0002 1.82 3 + mixed24-pca-w int8 1 33.2 +0.0154 +0.032 +0.0020 1.56 10 + mixed24-pca-w int8 2 53.5 +0.0107 +0.030 +0.0012 1.65 6 + mixed24-pca-w int8 3 76.0 +0.0039 +0.039 +0.0002 1.82 3 + mixed48 f32 1 25.6 +0.0181 +0.056 +0.0010 1.51 10 + mixed48 f32 2 53.2 +0.0122 +0.033 -0.0001 1.69 9 + mixed48 f32 3 56.5 +0.0051 +0.013 -0.0001 1.90 2 + mixed48 int8 1 25.6 +0.0182 +0.051 +0.0009 1.51 10 + mixed48 int8 2 53.2 +0.0116 +0.030 -0.0002 1.69 9 + mixed48 int8 3 56.5 +0.0048 +0.035 -0.0001 1.90 2 + mixed48-pca f32 1 19.1 +0.0205 +0.061 +0.0007 1.42 10 + mixed48-pca f32 2 34.9 +0.0163 +0.030 +0.0001 1.60 10 + mixed48-pca f32 3 52.4 +0.0064 +0.008 +0.0001 1.71 8 + mixed48-pca int8 1 19.1 +0.0203 +0.051 +0.0009 1.42 10 + mixed48-pca int8 2 34.9 +0.0155 +0.030 +0.0001 1.60 10 + mixed48-pca int8 3 52.4 +0.0059 +0.015 +0.0001 1.71 8 + mixed48-pca-w f32 1 19.4 +0.0220 +0.046 -0.0012 1.42 10 + mixed48-pca-w f32 2 36.0 +0.0133 +0.016 -0.0003 1.59 10 + mixed48-pca-w f32 3 54.5 +0.0056 +0.003 +0.0001 1.67 9 + mixed48-pca-w int8 1 19.4 +0.0218 +0.036 -0.0013 1.42 10 + mixed48-pca-w int8 2 36.0 +0.0125 +0.015 -0.0003 1.59 10 + mixed48-pca-w int8 3 54.5 +0.0050 +0.007 +0.0000 1.67 9 + mixed96-pca-w f32 1 13.6 +0.0315 +0.063 +nan 1.29 10 + mixed96-pca-w f32 2 17.6 +0.0222 +0.059 -0.0002 1.39 10 + mixed96-pca-w f32 3 22.9 +0.0136 +0.039 +0.0003 1.49 10 + mixed96-pca-w int8 1 13.6 +0.0307 +0.061 +nan 1.29 10 + mixed96-pca-w int8 2 17.6 +0.0206 +0.028 -0.0004 1.39 10 + mixed96-pca-w int8 3 22.9 +0.0121 +0.016 +0.0003 1.49 10 + pq24 f32 1 35.6 +0.0107 +0.029 +0.0018 1.60 9 + pq24 f32 2 60.7 +0.0104 +0.012 +0.0002 1.94 4 + pq24 f32 3 60.6 +0.0048 +0.000 +0.0005 2.09 1 + pq24 int8 1 35.6 +0.0110 +0.029 +0.0020 1.60 9 + pq24 int8 2 60.7 +0.0104 +0.013 +0.0001 1.94 4 + pq24 int8 3 60.6 +0.0048 +0.000 +0.0003 2.09 1 + pq48 f32 1 20.8 +0.0199 +0.047 +0.0014 1.47 10 + pq48 f32 2 40.9 +0.0135 +0.027 +0.0002 1.69 10 + pq48 f32 3 64.7 +0.0045 +0.020 +0.0001 1.78 8 + pq48 int8 1 20.8 +0.0197 +0.052 +0.0016 1.47 10 + pq48 int8 2 40.9 +0.0126 +0.032 +0.0002 1.69 10 + pq48 int8 3 64.7 +0.0040 +0.033 +0.0001 1.78 8 + pq96 f32 1 13.9 +0.0328 +0.043 +nan 1.30 10 + pq96 f32 2 18.3 +0.0189 +0.041 +0.0004 1.43 10 + pq96 f32 3 24.2 +0.0108 +0.041 -0.0001 1.52 10 + pq96 int8 1 13.9 +0.0324 +0.060 +nan 1.30 10 + pq96 int8 2 18.3 +0.0177 +0.042 -0.0000 1.43 10 + pq96 int8 3 24.2 +0.0095 +0.057 -0.0002 1.52 10 + overall: Δrecall +0.0151, Δp10 +0.037; adaptive ahead in 522 of 526 cells + +4. Benefits and drawbacks against the plain 48-byte binary scan (binary, none) + baseline: recall 0.6685, p10 0.410, ndcg loss 0.0240, 48 B/vector, read 0.0312 of an f32 scan + uniform scan, no re-score best: pq96 none none: recall 0.8704 (+0.2019), p10 0.730, ndcg loss 0.0031, ram 96 B + disk 0 B, read 0.0625, mean depth 0 (p90 0) + adaptive bits, no re-score best: mixed96-pca-w none none: recall 0.8855 (+0.2170), p10 0.740, ndcg loss 0.0022, ram 96 B + disk 0 B, read 0.0625, mean depth 0 (p90 0) + hybrid fixed depth, int8 best: mixed96-pca-w int8 fixed 100: recall 0.9905 (+0.3220), p10 0.970, ndcg loss 0.0001, ram 96 B + disk 392 B, read 0.0647, mean depth 100 (p90 100) + hybrid fixed depth, f32 best: mixed96-pca-w f32 fixed 100: recall 1.0000 (+0.3315), p10 1.000, ndcg loss 0.0000, ram 96 B + disk 1536 B, read 0.0712, mean depth 100 (p90 100) + hybrid adaptive depth, int8 best: pq96 int8 margin 3σ: recall 0.9890 (+0.3205), p10 0.970, ndcg loss -0.0000, ram 96 B + disk 392 B, read 0.0630, mean depth 24 (p90 37) + hybrid adaptive depth, f32 best: pq96 f32 margin 3σ: recall 0.9982 (+0.3296), p10 1.000, ndcg loss -0.0000, ram 96 B + disk 1536 B, read 0.0645, mean depth 24 (p90 37) diff --git a/experiments/2026-11-routing-oracle/README.md b/experiments/2026-11-routing-oracle/README.md new file mode 100644 index 0000000..745fb11 --- /dev/null +++ b/experiments/2026-11-routing-oracle/README.md @@ -0,0 +1,122 @@ +# Routing: how much can choosing the compression per query or per collection gain? + +The adaptive-depth policy in the [hybrid × adaptive matrix](../2026-11-hybrid-adaptive-matrix/) +is a simple router: each query gets as many re-scored candidates as its scan scores suggest it +needs. The natural generalisation is to route each query, or each collection, to whichever form +of compression serves it best, the way a mixture-of-experts gate routes tokens to experts. Before +building a better router, this experiment measures how much there is to gain, and how much of it +a router can actually see. + +**Hypothesis.** Per-query routing has real headroom, because the depth a query needs varies a lot +between queries. But a router only sees the scan's own scores, which are noisy, so a router +built on them will capture a small share of that headroom. Per-collection routing, choosing a +codec and a depth once from the noise-to-spacing curve, will get close to the best choice. + +**Result.** The first two are confirmed. The third holds for the codec choice and fails for the +depth. +- **The headroom is real:** a hindsight oracle beats a fixed depth by 5.5 to 7 points of + recall@10 at the same mean depth. The depth a query needs for full recall varies 6.3× between + the 10th and 90th percentile. +- **Almost none of it is visible.** The band a router can read from the scan's own scores tracks + the true band with Spearman 0.52 to 0.70. The margin policy captures 0 to 6% of the headroom. + A router that sets depth from noise ÷ band_hat does *worse* than a fixed depth at every budget + up to 50 candidates. +- **Per-collection routing picks the right codec, but not the right depth.** Predicting each + collection's cheapest scan codec and depth from the curve, fitted on the other corpora, picks + the same scan size as the best choice in 10 of 10 collections at 90% and 95% recall. Its depth, + though, is coarse. Aiming for 95%, the recall it actually reaches ranges from 0.86 to 1.00, and + aiming for 90%, from 0.71 to 0.99. So the hypothesis holds for the codec choice and fails for + the depth. A deployed version would have to measure the depth on a sample, not predict it. + +## Setup + +| | | +|---|---| +| Collections | The ten from [neighbour spacing](../2026-11-neighbour-spacing/): five corpora × bge-small, MiniLM | +| Queries | Each set's judged test queries: 38,124 query traces over 60 (collection, scan codec) pairs | +| Scan codecs | `binary`, `binary-centred`, `pq48`, `pq24`, `mixed48-pca-w`, `mixed24-pca-w` | +| Second stage | f32 (exact), so recall at depth N is the share of the true top 10 the scan ranks within N | +| Traces | `bench rescore --trace`: per query, the scan's rank of each true neighbour, the exact band and the band read from the scan's own scores (`band_hat`), and the margin policy's depth from 0.25σ to 6σ | +| Policies | fixed depth; margin (λσ below the scan's 10th score, σ calibrated on self-queries); ratio router, depth = a·k·(σ ÷ band_hat)^β with β = 1 and 1.3, swept over a; oracle, the best per-query depths for each mean-depth budget, solved exactly (Lagrangian over each query's steps) | +| Comparison | Every policy traces a curve of mean depth against mean recall@10; curves are compared at mean depths of 15, 20, 30, 50 and 100 | +| Per-collection routing | For targets of 90/95/99%: fit depth/k against noise ÷ band on four corpora, predict each scan codec's depth on the fifth, pick the cheapest (scan bytes × vectors + depth × 1,536 B), and check its real recall | +| Code | hvec commit `cd1864a` (`--trace`) | +| Command | `./run.sh` (records the traces), then `python3 -I analyse.py results/traces 10` | + +## Results + +### Per-query routing at equal mean depth + +Means over the 60 traces with every policy defined at that budget. Δ is against a fixed depth. + +| mean depth | fixed | margin | ratio router (best β) | oracle | headroom | margin captures | worst decile, fixed → margin | +|---:|---:|---:|---:|---:|---:|---:|---| +| 15 | 0.796 | −0.002 | −0.025 | +0.067 | 0.067 | −4% | 0.596 → 0.615 | +| 20 | 0.832 | −0.000 | −0.020 | +0.071 | 0.071 | 0% | 0.638 → 0.661 | +| 30 | 0.875 | +0.003 | −0.008 | +0.069 | 0.069 | 4% | 0.690 → 0.730 | +| 50 | 0.900 | +0.004 | −0.001 | +0.067 | 0.067 | 5% | 0.738 → 0.757 | +| 100 | 0.939 | +0.003 | +0.000 | +0.055 | 0.055 | 6% | 0.786 → 0.831 | + +The number of traces falls with the budget (from 45 to 7), because the margin grid does not +reach large mean depths for the better codecs. Compared exactly at each margin's own mean +depth instead (all 60 traces), the margin policy gains +0.006 at 1σ, +0.006 at 2σ and +0.003 at +3σ. It is ahead in 156 of 180 comparisons, and worst-decile recall rises by 4.1, 2.0 and 0.7 +points. + +### What a router can see + +| scan codec | Spearman, band_hat vs exact band | Spearman, 1 / band_hat vs depth needed | +|---|---:|---:| +| `binary` | 0.61 | 0.56 | +| `binary-centred` | 0.63 | 0.55 | +| `pq48` | 0.70 | 0.58 | +| `pq24` | 0.55 | 0.53 | +| `mixed48-pca-w` | 0.68 | 0.52 | +| `mixed24-pca-w` | 0.52 | 0.48 | + +### Per-collection routing + +| target | same scan size as the best | reaches the target (within 0.005) | recall actually reached | cost vs the best choice | +|---|---:|---:|---|---:| +| 90% | 10 / 10 | 5 / 10 | 0.713 to 0.989 | ×1.00 | +| 95% | 10 / 10 | 4 / 10 | 0.861 to 0.997 | ×1.01 | +| 99% | 7 / 10 | 4 / 10 | 0.950 to 1.000 | ×1.06 | + +The cost ratio looks perfect partly because the predictions that undershoot also read less. + +Under this cost model (bytes read: the whole scan plus the re-scored f32 candidates) the best +choice is almost always the smallest scan with a deeper re-score. At 95% it is `mixed24-pca-w` +in 9 of 10 collections, at depths from 17 (ArguAna, MiniLM) to 111 (FiQA, bge-small), reading +1.7 to 2.5% of the bytes of an f32 scan. + +## Discussion + +**Where the oracle's headroom comes from.** The oracle knows, for each query, how far the scan +displaced its worst-placed true neighbour. Mostly that is chance: one neighbour's score happened +to be pushed down by the code's noise. Nothing in the scan's own output predicts it well, because +the same noise that displaced the neighbour is all the scan reports. The band a query's scores +show is a property of the query, and it predicts only part of the variation. + +**Why the ratio router loses to fixed depth.** It sets depth in proportion to 1 / band_hat, so its +errors in band_hat become large errors in depth. Some queries get far too many candidates, which +uses up the budget. The margin policy is more robust: it counts candidates near the 10th score +directly, rather than extrapolating from one gap. + +**What this means for routing.** +- **Per query:** a cheap margin policy is worth keeping for its worst-query gain, but a smarter + per-query router built on scan scores alone is not worth building. +- **Per collection:** the noise-to-spacing curve picks the right codec, but its depth is off by + up to 9 points of recall at a 95% target. The practical version of "route to the best + compression" chooses the codec from the curve, then measures the depth directly. With an exact + second stage, measuring takes one pass of self-queries with traces. +- **Next:** storage-side multiplexing, a layered code read to different depths per candidate, + remains open; it changes what hybrid costs on disk, which this experiment does not measure. + +**Caveats.** +- **The cost model counts bytes, not time.** The [zero-copy scan](../2026-11-zero-copy-scan/) + experiment shows that every asymmetric scan costs about the same per vector regardless of its + size. In time, the smallest scan does not win; only the symmetric `binary-hamming` scan is + faster. +- **The traces cover six scan codecs, with an f32 second stage only.** +- **Few collections.** Ten collections, two of each corpus, make a small sample for the + per-collection test. diff --git a/experiments/2026-11-routing-oracle/analyse.py b/experiments/2026-11-routing-oracle/analyse.py new file mode 100644 index 0000000..d68997b --- /dev/null +++ b/experiments/2026-11-routing-oracle/analyse.py @@ -0,0 +1,239 @@ +"""Replay depth-routing policies from `bench rescore --trace` output. + + python3 -I analyse.py + +With an exact (f32) second stage, a query's recall@k at depth N is the share of its true top-k +neighbours that the scan ranks within N, so every policy below is scored exactly from the traces: + + fixed the same depth N for every query + margin every candidate within λσ of the scan's own k-th score (σ calibrated on self-queries) + ratio depth = a·k·(σ / band_hat)^β, band_hat read from the scan's own scores (free at query time) + oracle the best per-query depths for a mean-depth budget, in hindsight (Lagrangian, exact) + +Each policy traces a curve of mean depth against mean recall@k. They are compared at equal mean +depth. Headroom = oracle − fixed; captured = (policy − fixed) / headroom. The second part asks the +per-collection question: predict each collection's cheapest (scan codec, fixed depth) for a recall +target from noise ÷ band, with the curve fitted on the other corpora, and check it. +""" + +import gzip +import json +import math +import sys +from collections import defaultdict +from pathlib import Path + +CAP = 1000 +BUDGETS = [15, 20, 30, 50, 100] +TARGETS = [0.90, 0.95, 0.99] + + +def mean(xs): + xs = list(xs) + return sum(xs) / len(xs) if xs else float("nan") + + +def pct(xs, p): + s = sorted(xs) + return s[min(len(s) - 1, max(0, math.ceil(p * len(s)) - 1))] + + +def spearman(a, b): + def rk(x): + o = sorted(range(len(x)), key=lambda i: x[i]) + r = [0.0] * len(x) + i = 0 + while i < len(o): + j = i + while j + 1 < len(o) and x[o[j + 1]] == x[o[i]]: + j += 1 + for t in range(i, j + 1): + r[o[t]] = (i + j) / 2 + i = j + 1 + return r + ra, rb = rk(a), rk(b) + ma, mb = mean(ra), mean(rb) + cov = sum((x - ma) * (y - mb) for x, y in zip(ra, rb)) + return cov / math.sqrt(sum((x - ma) ** 2 for x in ra) * sum((y - mb) ** 2 for y in rb)) + + +class Trace: + def __init__(self, line): + d = json.loads(line) + self.collection, self.first, self.k = d["collection"], d["first"], d["k"] + self.first_bytes, self.vectors, self.noise = d["first_bytes"], d["vectors"], d["calibrated_noise"] + self.margins = d["trace_margins"] + self.ranks, self.band, self.band_hat = d["ranks"], d["band"], d["band_hat"] + self.margin_depths = d["margin_depths"] + self.n = len(self.ranks) + + def recall(self, q, depth): + return sum(1 for r in self.ranks[q] if r <= depth) / self.k + + def evaluate(self, depths): + rec = [self.recall(q, d) for q, d in enumerate(depths)] + return dict(depth=mean(depths), recall=mean(rec), p10=pct(rec, 0.10), p90_depth=pct(depths, 0.90)) + + # Policy curves: lists of evaluate() results, ordered by mean depth. + def fixed_curve(self): + return [self.evaluate([d] * self.n) for d in sorted({self.k, 12, 15, 20, 25, 30, 40, 50, 70, 100, 150, 200, 300, 500, CAP})] + + def margin_curve(self): + return [self.evaluate([md[i] for md in self.margin_depths]) for i in range(len(self.margins))] + + def ratio_curve(self, beta): + out = [] + for a in [0.25 * 1.25 ** i for i in range(30)]: + depths = [] + for bh in self.band_hat: + r = self.noise / bh if bh > 0 else 50.0 + depths.append(int(min(CAP, max(self.k, round(a * self.k * r ** beta))))) + out.append(self.evaluate(depths)) + return sorted(out, key=lambda e: e["depth"]) + + def oracle_curve(self): + # Per query, the candidate depths are k and each true neighbour's rank (≤ CAP); recall is a + # step function of depth, so the optimum for a budget is on these points. For each + # multiplier λ every query maximises recall − λ·depth independently. + options = [] + for q in range(self.n): + pts = sorted({self.k} | {r for r in self.ranks[q] if self.k < r <= CAP}) + options.append([(d, self.recall(q, d)) for d in pts]) + out = [] + for lam in [0.0] + [10 ** (e / 8) for e in range(-48, 9)]: + depths = [] + for opts in options: + best = max(opts, key=lambda o: (o[1] - lam * o[0], -o[0])) + depths.append(best[0]) + out.append(self.evaluate(depths)) + return sorted(out, key=lambda e: e["depth"]) + + +def at_budget(curve, budget): + """Linear interpolation of recall and p10 along a curve at a mean depth.""" + for a, b in zip(curve, curve[1:]): + if a["depth"] <= budget <= b["depth"]: + t = 0.0 if b["depth"] == a["depth"] else (budget - a["depth"]) / (b["depth"] - a["depth"]) + return {key: a[key] + t * (b[key] - a[key]) for key in ("recall", "p10", "p90_depth")} + return None + + +def main(traces_dir: Path, k: int) -> None: + def lines(p): + text = gzip.decompress(p.read_bytes()).decode() if p.suffix == ".gz" else p.read_text() + return [l for l in text.splitlines() if l.strip()] + + files = sorted(traces_dir.glob("*.jsonl")) + sorted(traces_dir.glob("*.jsonl.gz")) + traces = [Trace(l) for p in files for l in lines(p)] + print(f"{len(traces)} (collection, scan codec) traces, {sum(t.n for t in traces)} query traces in all\n") + + # Does the scan's own band track the exact one, query by query? + print("1. Can a router see the band? Spearman, per trace, over queries") + rows = defaultdict(list) + for t in traces: + full = [max(r) for r in t.ranks] + rows[t.first].append((spearman(t.band_hat, t.band), spearman([1 / b if b > 0 else 1e9 for b in t.band_hat], full))) + for first, v in sorted(rows.items()): + print(f" {first:<15} band_hat vs exact band {mean(x[0] for x in v):.3f} 1/band_hat vs depth for full recall {mean(x[1] for x in v):.3f}") + print(" (how much per-query depth needed varies:", + f"median p90/p10 ratio of full-recall depth {mean(pct([max(r) for r in t.ranks], .9) / max(1, pct([max(r) for r in t.ranks], .1)) for t in traces):.1f}×)") + + print("\n2. Per-query routing at equal mean depth (means over traces). Δ against fixed depth") + print(f" {'budget':>6} {'fixed':>7} {'margin':>8} {'ratio β=1':>10} {'ratio β=1.3':>11} {'oracle':>8} " + f"{'headroom':>8} {'margin cap.':>11} {'ratio cap.':>10} {'p10 fixed→margin':>17}") + table = defaultdict(lambda: defaultdict(list)) + for t in traces: + curves = dict(fixed=t.fixed_curve(), margin=t.margin_curve(), r1=t.ratio_curve(1.0), + r13=t.ratio_curve(1.29), oracle=t.oracle_curve()) + for b in BUDGETS: + vals = {name: at_budget(c, b) for name, c in curves.items()} + if any(v is None for v in vals.values()): + continue + for name, v in vals.items(): + table[b][name].append(v) + table[b]["first"].append(t.first) + for b in BUDGETS: + row = table[b] + if not row["fixed"]: + continue + r = {name: mean(v["recall"] for v in row[name]) for name in ("fixed", "margin", "r1", "r13", "oracle")} + head = r["oracle"] - r["fixed"] + cap = lambda name: (r[name] - r["fixed"]) / head if head > 1e-9 else float("nan") + best_ratio = max(("r1", "r13"), key=lambda n: r[n]) + p10f, p10m = mean(v["p10"] for v in row["fixed"]), mean(v["p10"] for v in row["margin"]) + print(f" {b:>6} {r['fixed']:>7.4f} {r['margin'] - r['fixed']:>+8.4f} {r['r1'] - r['fixed']:>+10.4f} " + f"{r['r13'] - r['fixed']:>+11.4f} {r['oracle'] - r['fixed']:>+8.4f} {head:>8.4f} {cap('margin'):>11.0%} " + f"{cap(best_ratio):>10.0%} {p10f:>8.3f}→{p10m:.3f} (n={len(row['fixed'])})") + + print("\n by scan codec, at a mean depth of 30: fixed recall, then Δ for margin, best ratio, oracle") + b = 30 + by = defaultdict(lambda: defaultdict(list)) + for i, first in enumerate(table[b]["first"]): + for name in ("fixed", "margin", "r1", "r13", "oracle"): + by[first][name].append(table[b][name][i]["recall"]) + for first in sorted(by): + r = {n: mean(v) for n, v in by[first].items()} + print(f" {first:<15} fixed {r['fixed']:.4f} margin {r['margin'] - r['fixed']:+.4f} " + f"ratio {max(r['r1'], r['r13']) - r['fixed']:+.4f} oracle {r['oracle'] - r['fixed']:+.4f}") + print(" p90 depth at a mean of 30: fixed 30, margin " + f"{mean(v['p90_depth'] for v in table[b]['margin']):.0f}, oracle {mean(v['p90_depth'] for v in table[b]['oracle']):.0f}") + + # Per-collection routing: choose the scan codec and a fixed depth for a target recall. + print("\n3. Per-collection routing: the cheapest (scan codec, fixed depth) reaching a recall target") + print(" cost = bytes read per query = vectors × scan bytes + depth × 1536; RAM = scan bytes per vector") + by_col = defaultdict(list) + for t in traces: + by_col[t.collection].append(t) + if len({c.split("-")[0] for c in by_col}) < 2: + print(" (needs at least two corpora: the depth curve is fitted on the others)") + return + + def needed(t, target): + # Smallest fixed depth whose mean recall reaches the target (exact, from the ranks). + ranks = sorted(r for rs in t.ranks for r in rs) + need = math.ceil(target * len(ranks)) + return max(t.k, ranks[need - 1]) if need > 0 else t.k + + for target in TARGETS: + pts = [] + for col, ts in by_col.items(): + for t in ts: + ratio = t.noise / sorted(t.band)[len(t.band) // 2] + pts.append((col, t, ratio, needed(t, target))) + corpora = sorted({c.split("-")[0] for c in by_col}) + hits, regret, ram_match = 0, [], 0 + for col in sorted(by_col): + corpus = col.split("-")[0] + train = [p for p in pts if p[0].split("-")[0] != corpus] + xs = [math.log(p[2]) for p in train] + ys = [math.log(p[3] / k) for p in train] + mx, my = mean(xs), mean(ys) + beta = sum((x - mx) * (y - my) for x, y in zip(xs, ys)) / sum((x - mx) ** 2 for x in xs) + alpha = my - beta * mx + mine = [p for p in pts if p[0] == col] + cost = lambda t, d: t.vectors * t.first_bytes + d * 1536 + best = min(mine, key=lambda p: (cost(p[1], p[3]), p[1].first_bytes)) + pred = [] + for p in mine: + d = int(min(CAP, max(k, round(k * math.exp(alpha + beta * math.log(p[2])))))) + pred.append((p, d)) + chosen, d = min(pred, key=lambda x: (cost(x[0][1], x[1]), x[0][1].first_bytes)) + got = chosen[1].evaluate([d] * chosen[1].n)["recall"] + ok = got >= target - 0.005 + hits += ok + regret.append(cost(chosen[1], d) / cost(best[1], best[3])) + ram_match += chosen[1].first_bytes == best[1].first_bytes + print(f" target {target:.0%}: predicted choice reaches the target (within 0.005) in {hits}/{len(by_col)} collections; " + f"cost vs the best choice ×{mean(regret):.2f} on average; same scan size as the best in {ram_match}/{len(by_col)}") + print("\n best choice per collection at 95%:") + for col in sorted(by_col): + opts = [(t.vectors * t.first_bytes + needed(t, 0.95) * 1536, t.first, needed(t, 0.95), t.first_bytes) for t in by_col[col]] + opts.sort() + c, first, d, b = opts[0] + cheapest_ram = min(opts, key=lambda o: (o[3], o[0])) + print(f" {col:<20} {first:<15} depth {d:>4} read {c / (by_col[col][0].vectors * 1536):.4f} of f32 " + f"(smallest scan: {cheapest_ram[1]} at depth {cheapest_ram[2]})") + + +if __name__ == "__main__": + main(Path(sys.argv[1]), int(sys.argv[2])) diff --git a/experiments/2026-11-routing-oracle/results/summary.txt b/experiments/2026-11-routing-oracle/results/summary.txt new file mode 100644 index 0000000..0e3f5fd --- /dev/null +++ b/experiments/2026-11-routing-oracle/results/summary.txt @@ -0,0 +1,45 @@ +60 (collection, scan codec) traces, 38124 query traces in all + +1. Can a router see the band? Spearman, per trace, over queries + binary band_hat vs exact band 0.607 1/band_hat vs depth for full recall 0.558 + binary-centred band_hat vs exact band 0.634 1/band_hat vs depth for full recall 0.546 + mixed24-pca-w band_hat vs exact band 0.524 1/band_hat vs depth for full recall 0.478 + mixed48-pca-w band_hat vs exact band 0.684 1/band_hat vs depth for full recall 0.522 + pq24 band_hat vs exact band 0.548 1/band_hat vs depth for full recall 0.526 + pq48 band_hat vs exact band 0.697 1/band_hat vs depth for full recall 0.579 + (how much per-query depth needed varies: median p90/p10 ratio of full-recall depth 6.3×) + +2. Per-query routing at equal mean depth (means over traces). Δ against fixed depth + budget fixed margin ratio β=1 ratio β=1.3 oracle headroom margin cap. ratio cap. p10 fixed→margin + 15 0.7961 -0.0024 -0.0253 -0.0336 +0.0672 0.0672 -4% -38% 0.596→0.615 (n=45) + 20 0.8319 -0.0001 -0.0201 -0.0335 +0.0710 0.0710 -0% -28% 0.638→0.661 (n=40) + 30 0.8752 +0.0025 -0.0075 -0.0223 +0.0687 0.0687 4% -11% 0.690→0.730 (n=31) + 50 0.8998 +0.0035 -0.0006 -0.0111 +0.0667 0.0667 5% -1% 0.738→0.757 (n=16) + 100 0.9385 +0.0033 +0.0003 -0.0047 +0.0546 0.0546 6% 1% 0.786→0.831 (n=7) + + by scan codec, at a mean depth of 30: fixed recall, then Δ for margin, best ratio, oracle + binary fixed 0.8819 margin +0.0046 ratio -0.0045 oracle +0.0655 + binary-centred fixed 0.9022 margin +0.0067 ratio -0.0026 oracle +0.0662 + mixed24-pca-w fixed 0.8453 margin -0.0002 ratio -0.0098 oracle +0.0753 + mixed48-pca-w fixed 0.9279 margin +0.0046 ratio -0.0079 oracle +0.0614 + pq24 fixed 0.8332 margin -0.0032 ratio -0.0146 oracle +0.0741 + pq48 fixed 0.9283 margin +0.0069 ratio -0.0033 oracle +0.0600 + p90 depth at a mean of 30: fixed 30, margin 46, oracle 52 + +3. Per-collection routing: the cheapest (scan codec, fixed depth) reaching a recall target + cost = bytes read per query = vectors × scan bytes + depth × 1536; RAM = scan bytes per vector + target 90%: predicted choice reaches the target (within 0.005) in 5/10 collections; cost vs the best choice ×1.00 on average; same scan size as the best in 10/10 + target 95%: predicted choice reaches the target (within 0.005) in 4/10 collections; cost vs the best choice ×1.01 on average; same scan size as the best in 10/10 + target 99%: predicted choice reaches the target (within 0.005) in 4/10 collections; cost vs the best choice ×1.06 on average; same scan size as the best in 7/10 + + best choice per collection at 95%: + arguana-bge-small mixed24-pca-w depth 25 read 0.0179 of f32 (smallest scan: mixed24-pca-w at depth 25) + arguana-minilm mixed24-pca-w depth 17 read 0.0171 of f32 (smallest scan: mixed24-pca-w at depth 17) + cosqa-bge-small mixed24-pca-w depth 63 read 0.0187 of f32 (smallest scan: mixed24-pca-w at depth 63) + cosqa-minilm mixed24-pca-w depth 43 read 0.0177 of f32 (smallest scan: mixed24-pca-w at depth 43) + fiqa-bge-small mixed24-pca-w depth 111 read 0.0172 of f32 (smallest scan: mixed24-pca-w at depth 111) + fiqa-minilm mixed24-pca-w depth 61 read 0.0165 of f32 (smallest scan: mixed24-pca-w at depth 61) + nfcorpus-bge-small pq24 depth 106 read 0.0326 of f32 (smallest scan: pq24 at depth 106) + nfcorpus-minilm mixed24-pca-w depth 54 read 0.0243 of f32 (smallest scan: mixed24-pca-w at depth 54) + scifact-bge-small mixed24-pca-w depth 71 read 0.0245 of f32 (smallest scan: mixed24-pca-w at depth 71) + scifact-minilm mixed24-pca-w depth 45 read 0.0212 of f32 (smallest scan: mixed24-pca-w at depth 45) diff --git a/experiments/2026-11-routing-oracle/results/traces/arguana-bge-small.jsonl.gz b/experiments/2026-11-routing-oracle/results/traces/arguana-bge-small.jsonl.gz new file mode 100644 index 0000000..8c68300 Binary files /dev/null and b/experiments/2026-11-routing-oracle/results/traces/arguana-bge-small.jsonl.gz differ diff --git a/experiments/2026-11-routing-oracle/results/traces/arguana-minilm.jsonl.gz b/experiments/2026-11-routing-oracle/results/traces/arguana-minilm.jsonl.gz new file mode 100644 index 0000000..9e1cf09 Binary files /dev/null and b/experiments/2026-11-routing-oracle/results/traces/arguana-minilm.jsonl.gz differ diff --git a/experiments/2026-11-routing-oracle/results/traces/cosqa-bge-small.jsonl.gz b/experiments/2026-11-routing-oracle/results/traces/cosqa-bge-small.jsonl.gz new file mode 100644 index 0000000..1d36fae Binary files /dev/null and b/experiments/2026-11-routing-oracle/results/traces/cosqa-bge-small.jsonl.gz differ diff --git a/experiments/2026-11-routing-oracle/results/traces/cosqa-minilm.jsonl.gz b/experiments/2026-11-routing-oracle/results/traces/cosqa-minilm.jsonl.gz new file mode 100644 index 0000000..e3966ba Binary files /dev/null and b/experiments/2026-11-routing-oracle/results/traces/cosqa-minilm.jsonl.gz differ diff --git a/experiments/2026-11-routing-oracle/results/traces/fiqa-bge-small.jsonl.gz b/experiments/2026-11-routing-oracle/results/traces/fiqa-bge-small.jsonl.gz new file mode 100644 index 0000000..af2032b Binary files /dev/null and b/experiments/2026-11-routing-oracle/results/traces/fiqa-bge-small.jsonl.gz differ diff --git a/experiments/2026-11-routing-oracle/results/traces/fiqa-minilm.jsonl.gz b/experiments/2026-11-routing-oracle/results/traces/fiqa-minilm.jsonl.gz new file mode 100644 index 0000000..e607bdd Binary files /dev/null and b/experiments/2026-11-routing-oracle/results/traces/fiqa-minilm.jsonl.gz differ diff --git a/experiments/2026-11-routing-oracle/results/traces/nfcorpus-bge-small.jsonl.gz b/experiments/2026-11-routing-oracle/results/traces/nfcorpus-bge-small.jsonl.gz new file mode 100644 index 0000000..3965497 Binary files /dev/null and b/experiments/2026-11-routing-oracle/results/traces/nfcorpus-bge-small.jsonl.gz differ diff --git a/experiments/2026-11-routing-oracle/results/traces/nfcorpus-minilm.jsonl.gz b/experiments/2026-11-routing-oracle/results/traces/nfcorpus-minilm.jsonl.gz new file mode 100644 index 0000000..7fbadd5 Binary files /dev/null and b/experiments/2026-11-routing-oracle/results/traces/nfcorpus-minilm.jsonl.gz differ diff --git a/experiments/2026-11-routing-oracle/results/traces/scifact-bge-small.jsonl.gz b/experiments/2026-11-routing-oracle/results/traces/scifact-bge-small.jsonl.gz new file mode 100644 index 0000000..a40fc9e Binary files /dev/null and b/experiments/2026-11-routing-oracle/results/traces/scifact-bge-small.jsonl.gz differ diff --git a/experiments/2026-11-routing-oracle/results/traces/scifact-minilm.jsonl.gz b/experiments/2026-11-routing-oracle/results/traces/scifact-minilm.jsonl.gz new file mode 100644 index 0000000..9fa3e10 Binary files /dev/null and b/experiments/2026-11-routing-oracle/results/traces/scifact-minilm.jsonl.gz differ diff --git a/experiments/2026-11-routing-oracle/run.sh b/experiments/2026-11-routing-oracle/run.sh new file mode 100755 index 0000000..3645462 --- /dev/null +++ b/experiments/2026-11-routing-oracle/run.sh @@ -0,0 +1,36 @@ +#!/usr/bin/env bash +# Routing: how much is there to gain by choosing the compression per query or per collection, +# and how much of it a free router captures. Records per-query traces (where the scan ranks each +# true neighbour, the band the scan itself sees, the margin policy's depth on a 0.25σ grid) for +# six scan codecs on the ten collections, then replays fixed, margin, ratio-router and oracle +# policies offline. Retrieval only. Reuses ../2026-11-neighbour-spacing's collections (and +# builds them if missing). Needs: a built `hvec` (cargo build --release). +set -euo pipefail +HERE="$(cd "$(dirname "$0")" && pwd)" +ROOT="$HERE/../.." +HVEC="${HVEC:-$ROOT/target/release/hvec}" +SPACING="$HERE/../2026-11-neighbour-spacing" +SPACING_WORK="${SPACING_WORK:-$SPACING/work}" +K="${K:-10}" +CORPORA="${CORPORA:-scifact nfcorpus arguana cosqa fiqa}" +MODELS="${MODELS:-bge-small minilm}" +FIRST="${FIRST:-binary,binary-centred,pq48,pq24,mixed48-pca-w,mixed24-pca-w}" +mkdir -p "$HERE/results/traces" + +export HVEC_CONFIG="$SPACING_WORK/config.toml" +if [ ! -f "$HVEC_CONFIG" ]; then + WORK="$SPACING_WORK" CORPORA="$CORPORA" MODELS="$MODELS" "$SPACING/run.sh" +fi + +for c in $CORPORA; do + for m in $MODELS; do + col="$c-$m" + # The cells themselves are not needed: the traces replay any depth policy exactly. + "$HVEC" bench rescore --collection "$col" -k "$K" --codecs "$FIRST" --rescore f32 \ + --depths "$K" --margins 1 --max-depth 1000 --questions "$SPACING_WORK/$c.jsonl" \ + --trace "$HERE/results/traces/$col.jsonl" --no-record --json > /dev/null + echo "== $col done" + done +done +gzip -9f "$HERE"/results/traces/*.jsonl +python3 -I "$HERE/analyse.py" "$HERE/results/traces" "$K" | tee "$HERE/results/summary.txt" diff --git a/experiments/2026-11-zero-copy-scan/README.md b/experiments/2026-11-zero-copy-scan/README.md new file mode 100644 index 0000000..5438019 --- /dev/null +++ b/experiments/2026-11-zero-copy-scan/README.md @@ -0,0 +1,133 @@ +# Zero-copy scan: where a search spends its time, and which codecs are actually faster + +Every earlier experiment measured what compression costs in recall. This one measures what it +buys in time, which the README had so far declined to claim: "binary is no faster than int8 +here". The question has two parts. How much of a search's time is moving bytes rather than +scoring them, and does reading fewer bytes make a scan faster? + +**Hypothesis.** +1. Most of the original search's time is copying, not scoring. Every query read every row, + blob and text out of SQLite and allocated a vector per row. Scanning a contiguous buffer in + place will be several times faster. +2. Once copying is gone, the asymmetric codecs (an f32 query against compressed codes) will + still be no faster than f32. Their scoring is limited by computation, not by bytes. Only a + symmetric code, the query binarised too and compared with XOR and popcount, will scan faster. + +**Result.** Both confirmed. +- **Copying is most of the original cost.** The cached buffer searches 5.1× faster than the + original row scan on FiQA (14.7 against 74.7 ms) and 5.2× on SciFact. SQLite's own memory map + alone gives 2.7–2.9×. +- **Fewer bytes do not mean a faster scan, except for popcount.** `int8`, `binary`, `pq48`, + `pq24` and `mixed48-pca-w` all cost 200 to 490 ns per vector on one thread, against 207 ns for + f32. They use under 2% of the machine's memory bandwidth. `binary-hamming` costs 10 ns, 20× + less than f32. +- **The fast scan needs a re-score.** `binary-hamming` alone keeps only 0.40 of FiQA's top 10. A + hamming scan plus an f32 re-score reaches 95% recall with 369 candidates, for about 0.79 ms + against 14.8 ms for exact f32 on one thread. That is the first speed claim hvec can make: about + 19× faster at 95% recall@10, with nDCG within 0.0003 of f32 at 500 candidates. + +## Setup + +| | | +|---|---| +| Collections | `scifact-bge-small` (8,036 vectors) and `fiqa-bge-small` (71,274), 384 dims, from [neighbour spacing](../2026-11-neighbour-spacing/) | +| Queries | 50 stored vectors per collection, each excluded from its own ranking when recall is counted; median of 3 timed passes after one warm-up pass | +| Storage paths (f32) | SQLite rows copied (the original `Store::search`: every row and its text per query); SQLite rows with borrowed blobs, SQLite's memory map off and on; the cached contiguous buffer (`Store::search` now) | +| Codecs, scanned in place | `f32`, `int8`, `binary`, `binary-centred`, `binary-hamming`, `pq48`, `pq24`, `mixed48-pca-w`, at 1, 2, 4, 8 and 10 threads | +| Bandwidth ceiling | A sequential sum of a 1 GiB `u64` buffer at the same thread counts | +| Machine | Apple M4 Pro, 10 performance + 4 efficiency cores, 24 GiB, nothing else running; release build | +| Code | hvec commit `8f87908` | +| Command | `./run.sh` (`results/.json`, `results/summary.txt`, `results/machine.txt`); the re-score depths for `binary-hamming` are in `results/rescore-.json` | + +## Results + +### Storage path: the same exact f32 search (ms per query) + +| path | SciFact (8,036) | FiQA (71,274) | speed-up on FiQA | +|---|---:|---:|---:| +| SQLite rows, copied (the original search) | 8.75 | 74.70 | 1.0× | +| SQLite rows, blob borrowed in place | 6.41 | 54.50 | 1.4× | +| SQLite rows, borrowed, SQLite memory map on | 3.01 | 27.40 | 2.7× | +| cached contiguous buffer, text for the hits only | **1.68** | **14.68** | **5.1×** | + +Building the buffer once costs 47 ms for FiQA's 110 MB from SQLite (borrowed blobs, one copy), +or 8.6 ms from a flat code file read in one call. + +### Codec scan, in place, one thread (FiQA) + +| codec | bytes / vector | ms / query | ns / vector | GB/s | share of bandwidth | recall@10 alone | +|---|---:|---:|---:|---:|---:|---:| +| `f32` | 1,536 | 14.76 | 207 | 7.42 | 9.5% | 1.000 | +| `int8` | 392 | 18.64 | 261 | 1.50 | 1.9% | 0.978 | +| `binary` | 48 | 17.56 | 246 | 0.19 | 0.3% | 0.504 | +| `binary-centred` | 48 | 20.48 | 287 | 0.17 | 0.2% | 0.562 | +| `pq48` | 48 | 14.44 | 203 | 0.24 | 0.3% | 0.644 | +| `pq24` | 24 | 14.06 | 197 | 0.12 | 0.2% | 0.422 | +| `mixed48-pca-w` | 48 | 32.94 | 462 | 0.10 | 0.1% | 0.708 | +| `binary-hamming` | 48 | **0.72** | **10** | 4.76 | 6.1% | 0.372 | + +Memory bandwidth measured 78 GB/s on one thread and 238 GB/s on ten. Recall here is over +self-queries; the judged-query figures in the other experiments are the ones to compare +codecs by. + +### Thread scaling (FiQA, ms per query) + +| codec | 1 | 2 | 4 | 8 | 10 | speed-up at 10 | +|---|---:|---:|---:|---:|---:|---:| +| `f32` | 14.76 | 7.56 | 4.00 | 2.09 | 1.71 | 8.7× (64 GB/s, 27% of bandwidth) | +| `pq48` | 14.44 | 7.36 | 3.78 | 2.00 | 1.62 | 8.9× | +| `mixed48-pca-w` | 32.94 | 17.34 | 8.49 | 4.60 | 3.84 | 8.6× | +| `binary-hamming` | 0.72 | 0.42 | 0.25 | 0.18 | 0.17 | 4.3× | + +On SciFact, `binary-hamming` stops scaling at two threads. A whole scan takes 0.09 ms there, +and starting the threads costs about as much as the work. + +### The fast hybrid: `binary-hamming` scan, f32 re-score (one thread) + +Re-score cost from the measured f32 cost per vector (207 ns). Fetching the candidates is +treated as reading from memory; it is not timed separately. + +| collection | candidates for 90 / 95 / 99% recall@10 | time at 95% | exact f32 scan | speed-up at 95% | +|---|---|---:|---:|---:| +| SciFact | 115 / 215 / 627 | 0.13 ms | 1.65 ms | 12× | +| FiQA | 186 / 369 / 1,254 | 0.79 ms | 14.76 ms | 19× | + +For comparison, the same 95% target needs 105 candidates behind a `binary` scan and 44 behind +`pq48` on FiQA. Those hybrids read far fewer bytes, but their scans already take as long as +exact f32. + +## Discussion + +**The original search was mostly bookkeeping.** Reading every chunk's text on every query, and +allocating a vector per row, cost more than the scoring. The fix needs no unsafe code and no new +index: keep the codes in one buffer, score them in place, and read text for the hits only. SQLite's +memory map is a cheap partial version of the same thing. + +**A smaller code only helps when the scoring is as cheap as the read.** At 48 bytes per vector a +scan reads 3.4 MB for FiQA, which this machine streams in well under a millisecond. But +asymmetric scoring does a float operation per dimension whatever the code size: `binary` walks +384 bits and adds or subtracts a float for each. So it costs what f32 costs. Only popcount does +work in proportion to the bytes. Compression buys memory everywhere, and time only with symmetric +scoring. + +**Adaptive bits have a speed cost.** `mixed48-pca-w`, the best 48-byte scan for recall in the +[matrix](../2026-11-hybrid-adaptive-matrix/), is the slowest codec here: 2.2× f32 per vector, +because every dimension's code has to be unpacked from a bit stream of varying widths. A version +with byte-aligned groups, or a lookup table per group of dimensions as PQ uses, would close most +of that gap. + +**The memory-mapped file is the one zero-copy path left out.** The workspace forbids `unsafe`, and +mapping a file needs one `unsafe` call. Reading a flat file is already fast (8.6 ms for 110 MB), +so a map would mostly save memory and start-up time on collections far larger than these. + +**Caveats.** +- **Brute force only.** With an index (HNSW, IVF) the scan is not over every vector, and the + ranking of these paths changes. +- **The hybrid times are computed, not measured end to end.** The re-score cost assumes the f32 + candidates are in memory. From disk, hundreds of random 1.5 KB reads per query would dominate. +- **Scoring is portable Rust with no explicit SIMD.** A tuned f32 kernel would move every row of + the codec table, f32 most of all. +- **Recall in the codec table is over self-queries**, a harder and less realistic test than the + judged queries the other experiments use. +- **One machine.** Other machines will give different absolute numbers; the ratios between paths + and codecs are the result. diff --git a/experiments/2026-11-zero-copy-scan/results/fiqa-bge-small.json b/experiments/2026-11-zero-copy-scan/results/fiqa-bge-small.json new file mode 100644 index 0000000..48c9ad9 --- /dev/null +++ b/experiments/2026-11-zero-copy-scan/results/fiqa-bge-small.json @@ -0,0 +1,386 @@ +{ + "collection": "fiqa-bge-small", + "vectors": 71274, + "dimension": 384, + "queries": 50, + "k": 10, + "repeat": 3, + "paths": [ + { + "path": "sqlite rows, copied", + "ms_per_query": 74.70102331999999, + "note": "the original search: every row and its text copied out on every query" + }, + { + "path": "sqlite rows, borrowed", + "ms_per_query": 54.501525, + "note": "vector column only, each blob scored in place in SQLite's page buffer" + }, + { + "path": "sqlite rows, borrowed, mmap", + "ms_per_query": 27.397842500000003, + "note": "as above, with SQLite reading the file through its memory map" + }, + { + "path": "cached buffer (Store::search)", + "ms_per_query": 14.67806916, + "note": "codes scanned in place in one allocation; text read for the hits only" + } + ], + "loads": [ + { + "source": "sqlite (borrowed blobs, one copy)", + "ms": 46.746415999999996, + "bytes": 109476864 + }, + { + "source": "flat code file (one read)", + "ms": 8.648792, + "bytes": 109476864 + } + ], + "codecs": [ + { + "codec": "f32", + "bytes_per_vector": 1536, + "threads": 1, + "ms_per_query": 14.762140839999999, + "gb_per_s": 7.416056057625312, + "recall_at_k": 1.0 + }, + { + "codec": "f32", + "bytes_per_vector": 1536, + "threads": 2, + "ms_per_query": 7.555813340000001, + "gb_per_s": 14.489090594712865, + "recall_at_k": 1.0 + }, + { + "codec": "f32", + "bytes_per_vector": 1536, + "threads": 4, + "ms_per_query": 3.9961374999999997, + "gb_per_s": 27.39566994378947, + "recall_at_k": 1.0 + }, + { + "codec": "f32", + "bytes_per_vector": 1536, + "threads": 8, + "ms_per_query": 2.0923091799999995, + "gb_per_s": 52.323463972948794, + "recall_at_k": 1.0 + }, + { + "codec": "f32", + "bytes_per_vector": 1536, + "threads": 10, + "ms_per_query": 1.7055316600000001, + "gb_per_s": 64.18928863507583, + "recall_at_k": 1.0 + }, + { + "codec": "int8", + "bytes_per_vector": 392, + "threads": 1, + "ms_per_query": 18.63756416, + "gb_per_s": 1.4990911773741145, + "recall_at_k": 0.978 + }, + { + "codec": "int8", + "bytes_per_vector": 392, + "threads": 2, + "ms_per_query": 9.54470416, + "gb_per_s": 2.927215713724122, + "recall_at_k": 0.978 + }, + { + "codec": "int8", + "bytes_per_vector": 392, + "threads": 4, + "ms_per_query": 4.8968275, + "gb_per_s": 5.70561409402312, + "recall_at_k": 0.978 + }, + { + "codec": "int8", + "bytes_per_vector": 392, + "threads": 8, + "ms_per_query": 2.5602358400000003, + "gb_per_s": 10.912825905913415, + "recall_at_k": 0.978 + }, + { + "codec": "int8", + "bytes_per_vector": 392, + "threads": 10, + "ms_per_query": 2.0892375, + "gb_per_s": 13.373016710642041, + "recall_at_k": 0.978 + }, + { + "codec": "binary", + "bytes_per_vector": 48, + "threads": 1, + "ms_per_query": 17.555124160000002, + "gb_per_s": 0.1948805356669149, + "recall_at_k": 0.504 + }, + { + "codec": "binary", + "bytes_per_vector": 48, + "threads": 2, + "ms_per_query": 8.919795, + "gb_per_s": 0.3835460344099836, + "recall_at_k": 0.504 + }, + { + "codec": "binary", + "bytes_per_vector": 48, + "threads": 4, + "ms_per_query": 4.58119334, + "gb_per_s": 0.7467818417809889, + "recall_at_k": 0.504 + }, + { + "codec": "binary", + "bytes_per_vector": 48, + "threads": 8, + "ms_per_query": 2.4290616800000002, + "gb_per_s": 1.4084253307227668, + "recall_at_k": 0.504 + }, + { + "codec": "binary", + "bytes_per_vector": 48, + "threads": 10, + "ms_per_query": 1.96065, + "gb_per_s": 1.7449070461326601, + "recall_at_k": 0.504 + }, + { + "codec": "binary-centred", + "bytes_per_vector": 48, + "threads": 1, + "ms_per_query": 20.47594916, + "gb_per_s": 0.1670814853693454, + "recall_at_k": 0.562 + }, + { + "codec": "binary-centred", + "bytes_per_vector": 48, + "threads": 2, + "ms_per_query": 10.372313339999998, + "gb_per_s": 0.32983500284421613, + "recall_at_k": 0.562 + }, + { + "codec": "binary-centred", + "bytes_per_vector": 48, + "threads": 4, + "ms_per_query": 5.340729179999999, + "gb_per_s": 0.6405776972948852, + "recall_at_k": 0.562 + }, + { + "codec": "binary-centred", + "bytes_per_vector": 48, + "threads": 8, + "ms_per_query": 2.7814266799999996, + "gb_per_s": 1.2299989874261221, + "recall_at_k": 0.562 + }, + { + "codec": "binary-centred", + "bytes_per_vector": 48, + "threads": 10, + "ms_per_query": 2.29705584, + "gb_per_s": 1.4893638806795397, + "recall_at_k": 0.562 + }, + { + "codec": "binary-hamming", + "bytes_per_vector": 48, + "threads": 1, + "ms_per_query": 0.71837084, + "gb_per_s": 4.762375933856113, + "recall_at_k": 0.372 + }, + { + "codec": "binary-hamming", + "bytes_per_vector": 48, + "threads": 2, + "ms_per_query": 0.415375, + "gb_per_s": 8.236297321697261, + "recall_at_k": 0.372 + }, + { + "codec": "binary-hamming", + "bytes_per_vector": 48, + "threads": 4, + "ms_per_query": 0.24892999999999998, + "gb_per_s": 13.743429879885914, + "recall_at_k": 0.372 + }, + { + "codec": "binary-hamming", + "bytes_per_vector": 48, + "threads": 8, + "ms_per_query": 0.17952500000000002, + "gb_per_s": 19.05668848349812, + "recall_at_k": 0.372 + }, + { + "codec": "binary-hamming", + "bytes_per_vector": 48, + "threads": 10, + "ms_per_query": 0.16839834, + "gb_per_s": 20.31582971661122, + "recall_at_k": 0.372 + }, + { + "codec": "pq48", + "bytes_per_vector": 48, + "threads": 1, + "ms_per_query": 14.4423525, + "gb_per_s": 0.23688329169364894, + "recall_at_k": 0.644 + }, + { + "codec": "pq48", + "bytes_per_vector": 48, + "threads": 2, + "ms_per_query": 7.364075839999999, + "gb_per_s": 0.4645731622448908, + "recall_at_k": 0.644 + }, + { + "codec": "pq48", + "bytes_per_vector": 48, + "threads": 4, + "ms_per_query": 3.7834966599999995, + "gb_per_s": 0.9042302154430871, + "recall_at_k": 0.644 + }, + { + "codec": "pq48", + "bytes_per_vector": 48, + "threads": 8, + "ms_per_query": 1.9979866600000002, + "gb_per_s": 1.7122997207598973, + "recall_at_k": 0.644 + }, + { + "codec": "pq48", + "bytes_per_vector": 48, + "threads": 10, + "ms_per_query": 1.6209075000000002, + "gb_per_s": 2.110639873034087, + "recall_at_k": 0.644 + }, + { + "codec": "pq24", + "bytes_per_vector": 24, + "threads": 1, + "ms_per_query": 14.055774999999999, + "gb_per_s": 0.12169915924237547, + "recall_at_k": 0.422 + }, + { + "codec": "pq24", + "bytes_per_vector": 24, + "threads": 2, + "ms_per_query": 7.25184666, + "gb_per_s": 0.23588143547425752, + "recall_at_k": 0.422 + }, + { + "codec": "pq24", + "bytes_per_vector": 24, + "threads": 4, + "ms_per_query": 3.78886416, + "gb_per_s": 0.4514746181874201, + "recall_at_k": 0.422 + }, + { + "codec": "pq24", + "bytes_per_vector": 24, + "threads": 8, + "ms_per_query": 1.9864024999999998, + "gb_per_s": 0.8611426938900854, + "recall_at_k": 0.422 + }, + { + "codec": "pq24", + "bytes_per_vector": 24, + "threads": 10, + "ms_per_query": 1.6198083399999998, + "gb_per_s": 1.0560360493019811, + "recall_at_k": 0.422 + }, + { + "codec": "mixed48-pca-w", + "bytes_per_vector": 48, + "threads": 1, + "ms_per_query": 32.93501834, + "gb_per_s": 0.10387581888317844, + "recall_at_k": 0.708 + }, + { + "codec": "mixed48-pca-w", + "bytes_per_vector": 48, + "threads": 2, + "ms_per_query": 17.341954180000002, + "gb_per_s": 0.19727603731911142, + "recall_at_k": 0.708 + }, + { + "codec": "mixed48-pca-w", + "bytes_per_vector": 48, + "threads": 4, + "ms_per_query": 8.489275, + "gb_per_s": 0.40299695792632473, + "recall_at_k": 0.708 + }, + { + "codec": "mixed48-pca-w", + "bytes_per_vector": 48, + "threads": 8, + "ms_per_query": 4.59580332, + "gb_per_s": 0.7444078351029173, + "recall_at_k": 0.708 + }, + { + "codec": "mixed48-pca-w", + "bytes_per_vector": 48, + "threads": 10, + "ms_per_query": 3.839975, + "gb_per_s": 0.8909308003307314, + "recall_at_k": 0.708 + } + ], + "bandwidth": [ + { + "threads": 1, + "gb_per_s": 77.78271917417484 + }, + { + "threads": 2, + "gb_per_s": 142.4131603362236 + }, + { + "threads": 4, + "gb_per_s": 148.250571122847 + }, + { + "threads": 8, + "gb_per_s": 208.80437248922377 + }, + { + "threads": 10, + "gb_per_s": 238.32240700472343 + } + ] +} diff --git a/experiments/2026-11-zero-copy-scan/results/machine.txt b/experiments/2026-11-zero-copy-scan/results/machine.txt new file mode 100644 index 0000000..cddb9e0 --- /dev/null +++ b/experiments/2026-11-zero-copy-scan/results/machine.txt @@ -0,0 +1,4 @@ +hvec 8f87908 +Apple M4 Pro +performance and efficiency cores: 10 4 +memory: 26 GB diff --git a/experiments/2026-11-zero-copy-scan/results/rescore-fiqa-bge-small.json b/experiments/2026-11-zero-copy-scan/results/rescore-fiqa-bge-small.json new file mode 100644 index 0000000..e41dae1 --- /dev/null +++ b/experiments/2026-11-zero-copy-scan/results/rescore-fiqa-bge-small.json @@ -0,0 +1,768 @@ +{ + "cells": [ + { + "bytes_read_per_query": 3421152.0, + "depth": 0, + "doc_recall_at_k": 0.33069296958185856, + "first": "binary-hamming", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.2845371521252547, + "p10_recall_at_k": 0.2, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 648, + "recall_at_k": 0.40200617283950646, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.375, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3497952.0, + "depth": 50, + "doc_recall_at_k": 0.4156827097799321, + "first": "binary-hamming", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.37435438444608504, + "p10_recall_at_k": 0.5, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.7206790123456781, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9290123456790124, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3574752.0, + "depth": 100, + "doc_recall_at_k": 0.423161059619393, + "first": "binary-hamming", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.3779816657308734, + "p10_recall_at_k": 0.6, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.8322530864197525, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9660493827160493, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3728352.0, + "depth": 200, + "doc_recall_at_k": 0.42573736277439983, + "first": "binary-hamming", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 200.0, + "metric": "cosine", + "ndcg_at_k": 0.3797903277843906, + "p10_recall_at_k": 0.7, + "p90_depth": 200, + "policy": { + "depth": 200, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.906635802469134, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9861111111111112, + "vectors": 71274 + }, + { + "bytes_read_per_query": 4189152.0, + "depth": 500, + "doc_recall_at_k": 0.43675845330937924, + "first": "binary-hamming", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 500.0, + "metric": "cosine", + "ndcg_at_k": 0.38670419957503743, + "p10_recall_at_k": 0.9, + "p90_depth": 500, + "policy": { + "depth": 500, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.9635802469135779, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9969135802469136, + "vectors": 71274 + }, + { + "bytes_read_per_query": 4957152.0, + "depth": 1000, + "doc_recall_at_k": 0.4389446673011487, + "first": "binary-hamming", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 1000.0, + "metric": "cosine", + "ndcg_at_k": 0.3873766509988365, + "p10_recall_at_k": 0.9, + "p90_depth": 1000, + "policy": { + "depth": 1000, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.9847222222222212, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9984567901234568, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3507063.703703704, + "depth": 56, + "doc_recall_at_k": 0.41369735802606183, + "first": "binary-hamming", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 55.9320987654321, + "metric": "cosine", + "ndcg_at_k": 0.3731143678602058, + "p10_recall_at_k": 0.5, + "p90_depth": 87, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 648, + "recall_at_k": 0.7367283950617277, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9506172839506173, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3812497.777777778, + "depth": 255, + "doc_recall_at_k": 0.4352924039266633, + "first": "binary-hamming", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 254.78240740740742, + "metric": "cosine", + "ndcg_at_k": 0.38585741744555463, + "p10_recall_at_k": 0.8, + "p90_depth": 448, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 648, + "recall_at_k": 0.9316358024691329, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9953703703703703, + "vectors": 71274 + }, + { + "bytes_read_per_query": 4547217.777777778, + "depth": 733, + "doc_recall_at_k": 0.4389446673011487, + "first": "binary-hamming", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 733.1157407407408, + "metric": "cosine", + "ndcg_at_k": 0.3873874156422938, + "p10_recall_at_k": 0.9, + "p90_depth": 1000, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 648, + "recall_at_k": 0.9819444444444432, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9984567901234568, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3421152.0, + "depth": 0, + "doc_recall_at_k": 0.3947765026468731, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.3334523978137343, + "p10_recall_at_k": 0.3, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 648, + "recall_at_k": 0.5367283950617282, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.49537037037037035, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3497952.0, + "depth": 50, + "doc_recall_at_k": 0.43210983974872874, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.3834727426221282, + "p10_recall_at_k": 0.7, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.8793209876543183, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9861111111111112, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3574752.0, + "depth": 100, + "doc_recall_at_k": 0.43637265084024335, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.3866850426308114, + "p10_recall_at_k": 0.8, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.9478395061728367, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9984567901234568, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3728352.0, + "depth": 200, + "doc_recall_at_k": 0.43666659557863263, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 200.0, + "metric": "cosine", + "ndcg_at_k": 0.3864700858645261, + "p10_recall_at_k": 0.9, + "p90_depth": 200, + "policy": { + "depth": 200, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.9811728395061717, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 71274 + }, + { + "bytes_read_per_query": 4189152.0, + "depth": 500, + "doc_recall_at_k": 0.438503750193565, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 500.0, + "metric": "cosine", + "ndcg_at_k": 0.38702083481577326, + "p10_recall_at_k": 1.0, + "p90_depth": 500, + "policy": { + "depth": 500, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.9949074074074069, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 71274 + }, + { + "bytes_read_per_query": 4957152.0, + "depth": 1000, + "doc_recall_at_k": 0.438503750193565, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 1000.0, + "metric": "cosine", + "ndcg_at_k": 0.38702083481577326, + "p10_recall_at_k": 1.0, + "p90_depth": 1000, + "policy": { + "depth": 1000, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.9993827160493828, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3468922.074074074, + "depth": 31, + "doc_recall_at_k": 0.4183023980477685, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 31.10030864197531, + "metric": "cosine", + "ndcg_at_k": 0.37742668658172396, + "p10_recall_at_k": 0.6, + "p90_depth": 45, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 648, + "recall_at_k": 0.8061728395061714, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9814814814814815, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3563002.074074074, + "depth": 92, + "doc_recall_at_k": 0.4366408754140237, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 92.3503086419753, + "metric": "cosine", + "ndcg_at_k": 0.3863786242657091, + "p10_recall_at_k": 0.8, + "p90_depth": 152, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 648, + "recall_at_k": 0.9475308641975275, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9984567901234568, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3806104.888888889, + "depth": 251, + "doc_recall_at_k": 0.43666659557863263, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 250.62037037037038, + "metric": "cosine", + "ndcg_at_k": 0.3864520107139816, + "p10_recall_at_k": 1.0, + "p90_depth": 453, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 648, + "recall_at_k": 0.9899691358024683, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3421152.0, + "depth": 0, + "doc_recall_at_k": 0.4089782259921148, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.3532630384905834, + "p10_recall_at_k": 0.4, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 648, + "recall_at_k": 0.6418209876543218, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.5925925925925926, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3497952.0, + "depth": 50, + "doc_recall_at_k": 0.4366518983417132, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.38628352663861104, + "p10_recall_at_k": 0.9, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.9646604938271585, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3574752.0, + "depth": 100, + "doc_recall_at_k": 0.438503750193565, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.3870295309417207, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.9929012345679007, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3728352.0, + "depth": 200, + "doc_recall_at_k": 0.438503750193565, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 200.0, + "metric": "cosine", + "ndcg_at_k": 0.38702083481577326, + "p10_recall_at_k": 1.0, + "p90_depth": 200, + "policy": { + "depth": 200, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 0.9987654320987653, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 71274 + }, + { + "bytes_read_per_query": 4189152.0, + "depth": 500, + "doc_recall_at_k": 0.438503750193565, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 500.0, + "metric": "cosine", + "ndcg_at_k": 0.38702083481577326, + "p10_recall_at_k": 1.0, + "p90_depth": 500, + "policy": { + "depth": 500, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 1.0, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 71274 + }, + { + "bytes_read_per_query": 4957152.0, + "depth": 1000, + "doc_recall_at_k": 0.438503750193565, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 1000.0, + "metric": "cosine", + "ndcg_at_k": 0.38702083481577326, + "p10_recall_at_k": 1.0, + "p90_depth": 1000, + "policy": { + "depth": 1000, + "kind": "fixed" + }, + "queries": 648, + "recall_at_k": 1.0, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3457831.111111111, + "depth": 24, + "doc_recall_at_k": 0.43176935376009445, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 23.87962962962963, + "metric": "cosine", + "ndcg_at_k": 0.3839144126277606, + "p10_recall_at_k": 0.7, + "p90_depth": 34, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 648, + "recall_at_k": 0.8749999999999967, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9907407407407407, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3505797.925925926, + "depth": 55, + "doc_recall_at_k": 0.4362403757079683, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 55.108024691358025, + "metric": "cosine", + "ndcg_at_k": 0.386305810088604, + "p10_recall_at_k": 0.9, + "p90_depth": 89, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 648, + "recall_at_k": 0.9737654320987633, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 71274 + }, + { + "bytes_read_per_query": 3609899.8518518517, + "depth": 123, + "doc_recall_at_k": 0.438503750193565, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 648, + "mean_depth": 122.88271604938272, + "metric": "cosine", + "ndcg_at_k": 0.38702083481577326, + "p10_recall_at_k": 1.0, + "p90_depth": 220, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 648, + "recall_at_k": 0.9973765432098761, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 71274 + } + ], + "profiles": [ + { + "depth_for_recall": [ + [ + 0.9, + 186 + ], + [ + 0.95, + 369 + ], + [ + 0.99, + 1254 + ], + [ + 1.0, + 5879 + ] + ], + "first": "binary-hamming", + "first_bytes": 48, + "k": 10, + "median_full_depth": 196.5, + "metric": "cosine", + "neighbour_band": 0.010756641626358032, + "queries": 648, + "rank_noise": 0.0396927218576828, + "recall_at_k": 0.40200617283950646, + "vectors": 71274 + }, + { + "depth_for_recall": [ + [ + 0.9, + 60 + ], + [ + 0.95, + 105 + ], + [ + 0.99, + 327 + ], + [ + 1.0, + 1720 + ] + ], + "first": "binary", + "first_bytes": 48, + "k": 10, + "median_full_depth": 67.0, + "metric": "cosine", + "neighbour_band": 0.010756641626358032, + "queries": 648, + "rank_noise": 0.01769264290610194, + "recall_at_k": 0.5367283950617282, + "vectors": 71274 + }, + { + "depth_for_recall": [ + [ + 0.9, + 29 + ], + [ + 0.95, + 44 + ], + [ + 0.99, + 88 + ], + [ + 1.0, + 438 + ] + ], + "first": "pq48", + "first_bytes": 48, + "k": 10, + "median_full_depth": 34.0, + "metric": "cosine", + "neighbour_band": 0.010756641626358032, + "queries": 648, + "rank_noise": 0.010514919675474296, + "recall_at_k": 0.6418209876543218, + "vectors": 71274 + } + ] +} diff --git a/experiments/2026-11-zero-copy-scan/results/rescore-fiqa-bge-small.trace.jsonl.gz b/experiments/2026-11-zero-copy-scan/results/rescore-fiqa-bge-small.trace.jsonl.gz new file mode 100644 index 0000000..4ee8cc9 Binary files /dev/null and b/experiments/2026-11-zero-copy-scan/results/rescore-fiqa-bge-small.trace.jsonl.gz differ diff --git a/experiments/2026-11-zero-copy-scan/results/rescore-scifact-bge-small.json b/experiments/2026-11-zero-copy-scan/results/rescore-scifact-bge-small.json new file mode 100644 index 0000000..b4f628a --- /dev/null +++ b/experiments/2026-11-zero-copy-scan/results/rescore-scifact-bge-small.json @@ -0,0 +1,768 @@ +{ + "cells": [ + { + "bytes_read_per_query": 385728.0, + "depth": 0, + "doc_recall_at_k": 0.736111111111111, + "first": "binary-hamming", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.6067540263052055, + "p10_recall_at_k": 0.2, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 300, + "recall_at_k": 0.4753333333333332, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.5666666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 462528.0, + "depth": 50, + "doc_recall_at_k": 0.7895555555555556, + "first": "binary-hamming", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.6851192092966032, + "p10_recall_at_k": 0.5, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.8006666666666675, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.97, + "vectors": 8036 + }, + { + "bytes_read_per_query": 539328.0, + "depth": 100, + "doc_recall_at_k": 0.8062222222222223, + "first": "binary-hamming", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.6922197778386173, + "p10_recall_at_k": 0.7, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.887666666666668, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9866666666666667, + "vectors": 8036 + }, + { + "bytes_read_per_query": 692928.0, + "depth": 200, + "doc_recall_at_k": 0.8195555555555556, + "first": "binary-hamming", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 200.0, + "metric": "cosine", + "ndcg_at_k": 0.6987970935928796, + "p10_recall_at_k": 0.8, + "p90_depth": 200, + "policy": { + "depth": 200, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.9450000000000007, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 1153728.0, + "depth": 500, + "doc_recall_at_k": 0.8162222222222223, + "first": "binary-hamming", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 500.0, + "metric": "cosine", + "ndcg_at_k": 0.6973641975263443, + "p10_recall_at_k": 0.9, + "p90_depth": 500, + "policy": { + "depth": 500, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.9840000000000001, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 1921728.0, + "depth": 1000, + "doc_recall_at_k": 0.8178888888888889, + "first": "binary-hamming", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 1000.0, + "metric": "cosine", + "ndcg_at_k": 0.698128294524517, + "p10_recall_at_k": 1.0, + "p90_depth": 1000, + "policy": { + "depth": 1000, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.9973333333333333, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 452359.68, + "depth": 43, + "doc_recall_at_k": 0.7903888888888889, + "first": "binary-hamming", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 43.38, + "metric": "cosine", + "ndcg_at_k": 0.6861624150340037, + "p10_recall_at_k": 0.6, + "p90_depth": 67, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 300, + "recall_at_k": 0.7760000000000005, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 0.9733333333333334, + "vectors": 8036 + }, + { + "bytes_read_per_query": 631375.36, + "depth": 160, + "doc_recall_at_k": 0.8195555555555556, + "first": "binary-hamming", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 159.92666666666668, + "metric": "cosine", + "ndcg_at_k": 0.6986468252553855, + "p10_recall_at_k": 0.8, + "p90_depth": 281, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 300, + "recall_at_k": 0.9470000000000008, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 1109168.6400000001, + "depth": 471, + "doc_recall_at_k": 0.8178888888888889, + "first": "binary-hamming", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 470.99, + "metric": "cosine", + "ndcg_at_k": 0.6981764107949228, + "p10_recall_at_k": 1.0, + "p90_depth": 836, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 300, + "recall_at_k": 0.9916666666666668, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 385728.0, + "depth": 0, + "doc_recall_at_k": 0.7676111111111111, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.6536746843702901, + "p10_recall_at_k": 0.4, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 300, + "recall_at_k": 0.6109999999999995, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.7, + "vectors": 8036 + }, + { + "bytes_read_per_query": 462528.0, + "depth": 50, + "doc_recall_at_k": 0.8195555555555556, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.6985660154531242, + "p10_recall_at_k": 0.8, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.9193333333333347, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 539328.0, + "depth": 100, + "doc_recall_at_k": 0.8212222222222223, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.6993323364723006, + "p10_recall_at_k": 0.9, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.9700000000000002, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 692928.0, + "depth": 200, + "doc_recall_at_k": 0.8178888888888889, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 200.0, + "metric": "cosine", + "ndcg_at_k": 0.6982045407037659, + "p10_recall_at_k": 1.0, + "p90_depth": 200, + "policy": { + "depth": 200, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.991, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 1153728.0, + "depth": 500, + "doc_recall_at_k": 0.8178888888888889, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 500.0, + "metric": "cosine", + "ndcg_at_k": 0.698128294524517, + "p10_recall_at_k": 1.0, + "p90_depth": 500, + "policy": { + "depth": 500, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.9986666666666667, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 1921728.0, + "depth": 1000, + "doc_recall_at_k": 0.8178888888888889, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 1000.0, + "metric": "cosine", + "ndcg_at_k": 0.698128294524517, + "p10_recall_at_k": 1.0, + "p90_depth": 1000, + "policy": { + "depth": 1000, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 1.0, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 424727.04, + "depth": 25, + "doc_recall_at_k": 0.8135555555555556, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 25.39, + "metric": "cosine", + "ndcg_at_k": 0.6959960822987686, + "p10_recall_at_k": 0.7, + "p90_depth": 38, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 300, + "recall_at_k": 0.8236666666666677, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 482383.36, + "depth": 63, + "doc_recall_at_k": 0.8245555555555556, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 62.92666666666667, + "metric": "cosine", + "ndcg_at_k": 0.7004482246593918, + "p10_recall_at_k": 0.9, + "p90_depth": 108, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 300, + "recall_at_k": 0.9510000000000008, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 611013.12, + "depth": 147, + "doc_recall_at_k": 0.8178888888888889, + "first": "binary", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 146.67, + "metric": "cosine", + "ndcg_at_k": 0.698128294524517, + "p10_recall_at_k": 1.0, + "p90_depth": 275, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 300, + "recall_at_k": 0.9906666666666665, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 385728.0, + "depth": 0, + "doc_recall_at_k": 0.7968888888888889, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 0.0, + "metric": "cosine", + "ndcg_at_k": 0.6775694097419889, + "p10_recall_at_k": 0.5, + "p90_depth": 0, + "policy": { + "kind": "none" + }, + "queries": 300, + "recall_at_k": 0.711333333333333, + "rescore_ms": 0, + "second": "none", + "second_bytes": 0, + "top1_agreement": 0.8033333333333333, + "vectors": 8036 + }, + { + "bytes_read_per_query": 462528.0, + "depth": 50, + "doc_recall_at_k": 0.8145555555555556, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 50.0, + "metric": "cosine", + "ndcg_at_k": 0.6971051835816321, + "p10_recall_at_k": 0.9, + "p90_depth": 50, + "policy": { + "depth": 50, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.9836666666666667, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 539328.0, + "depth": 100, + "doc_recall_at_k": 0.8178888888888889, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 100.0, + "metric": "cosine", + "ndcg_at_k": 0.6981681784223372, + "p10_recall_at_k": 1.0, + "p90_depth": 100, + "policy": { + "depth": 100, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.9983333333333333, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 692928.0, + "depth": 200, + "doc_recall_at_k": 0.8178888888888889, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 200.0, + "metric": "cosine", + "ndcg_at_k": 0.698128294524517, + "p10_recall_at_k": 1.0, + "p90_depth": 200, + "policy": { + "depth": 200, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 0.9996666666666666, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 1153728.0, + "depth": 500, + "doc_recall_at_k": 0.8178888888888889, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 500.0, + "metric": "cosine", + "ndcg_at_k": 0.698128294524517, + "p10_recall_at_k": 1.0, + "p90_depth": 500, + "policy": { + "depth": 500, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 1.0, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 1921728.0, + "depth": 1000, + "doc_recall_at_k": 0.8178888888888889, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 1000.0, + "metric": "cosine", + "ndcg_at_k": 0.698128294524517, + "p10_recall_at_k": 1.0, + "p90_depth": 1000, + "policy": { + "depth": 1000, + "kind": "fixed" + }, + "queries": 300, + "recall_at_k": 1.0, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 418045.44, + "depth": 21, + "doc_recall_at_k": 0.8078888888888889, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 21.04, + "metric": "cosine", + "ndcg_at_k": 0.69556185762333, + "p10_recall_at_k": 0.8, + "p90_depth": 29, + "policy": { + "kind": "margin", + "sigmas": 1.0 + }, + "queries": 300, + "recall_at_k": 0.912333333333335, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 453271.04000000004, + "depth": 44, + "doc_recall_at_k": 0.8178888888888889, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 43.973333333333336, + "metric": "cosine", + "ndcg_at_k": 0.6982162946927432, + "p10_recall_at_k": 0.9, + "p90_depth": 71, + "policy": { + "kind": "margin", + "sigmas": 2.0 + }, + "queries": 300, + "recall_at_k": 0.9853333333333332, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + }, + { + "bytes_read_per_query": 523584.0, + "depth": 90, + "doc_recall_at_k": 0.8178888888888889, + "first": "pq48", + "first_bytes": 48, + "k": 10, + "labelled_queries": 300, + "mean_depth": 89.75, + "metric": "cosine", + "ndcg_at_k": 0.6981681784223372, + "p10_recall_at_k": 1.0, + "p90_depth": 154, + "policy": { + "kind": "margin", + "sigmas": 3.0 + }, + "queries": 300, + "recall_at_k": 0.9970000000000001, + "rescore_ms": 0, + "second": "f32", + "second_bytes": 1536, + "top1_agreement": 1.0, + "vectors": 8036 + } + ], + "profiles": [ + { + "depth_for_recall": [ + [ + 0.9, + 115 + ], + [ + 0.95, + 215 + ], + [ + 0.99, + 627 + ], + [ + 1.0, + 1798 + ] + ], + "first": "binary-hamming", + "first_bytes": 48, + "k": 10, + "median_full_depth": 108.0, + "metric": "cosine", + "neighbour_band": 0.013464927673339844, + "queries": 300, + "rank_noise": 0.044832806572036, + "recall_at_k": 0.4753333333333332, + "vectors": 8036 + }, + { + "depth_for_recall": [ + [ + 0.9, + 40 + ], + [ + 0.95, + 71 + ], + [ + 0.99, + 192 + ], + [ + 1.0, + 633 + ] + ], + "first": "binary", + "first_bytes": 48, + "k": 10, + "median_full_depth": 45.0, + "metric": "cosine", + "neighbour_band": 0.013464927673339844, + "queries": 300, + "rank_noise": 0.018715791052223443, + "recall_at_k": 0.6109999999999995, + "vectors": 8036 + }, + { + "depth_for_recall": [ + [ + 0.9, + 21 + ], + [ + 0.95, + 31 + ], + [ + 0.99, + 59 + ], + [ + 1.0, + 271 + ] + ], + "first": "pq48", + "first_bytes": 48, + "k": 10, + "median_full_depth": 23.0, + "metric": "cosine", + "neighbour_band": 0.013464927673339844, + "queries": 300, + "rank_noise": 0.011187321269100204, + "recall_at_k": 0.711333333333333, + "vectors": 8036 + } + ] +} diff --git a/experiments/2026-11-zero-copy-scan/results/rescore-scifact-bge-small.trace.jsonl.gz b/experiments/2026-11-zero-copy-scan/results/rescore-scifact-bge-small.trace.jsonl.gz new file mode 100644 index 0000000..2d308f3 Binary files /dev/null and b/experiments/2026-11-zero-copy-scan/results/rescore-scifact-bge-small.trace.jsonl.gz differ diff --git a/experiments/2026-11-zero-copy-scan/results/scifact-bge-small.json b/experiments/2026-11-zero-copy-scan/results/scifact-bge-small.json new file mode 100644 index 0000000..632d86c --- /dev/null +++ b/experiments/2026-11-zero-copy-scan/results/scifact-bge-small.json @@ -0,0 +1,386 @@ +{ + "collection": "scifact-bge-small", + "vectors": 8036, + "dimension": 384, + "queries": 50, + "k": 10, + "repeat": 3, + "paths": [ + { + "path": "sqlite rows, copied", + "ms_per_query": 8.75024418, + "note": "the original search: every row and its text copied out on every query" + }, + { + "path": "sqlite rows, borrowed", + "ms_per_query": 6.40946582, + "note": "vector column only, each blob scored in place in SQLite's page buffer" + }, + { + "path": "sqlite rows, borrowed, mmap", + "ms_per_query": 3.0057808400000003, + "note": "as above, with SQLite reading the file through its memory map" + }, + { + "path": "cached buffer (Store::search)", + "ms_per_query": 1.68215082, + "note": "codes scanned in place in one allocation; text read for the hits only" + } + ], + "loads": [ + { + "source": "sqlite (borrowed blobs, one copy)", + "ms": 6.5695, + "bytes": 12343296 + }, + { + "source": "flat code file (one read)", + "ms": 1.8046659999999999, + "bytes": 12343296 + } + ], + "codecs": [ + { + "codec": "f32", + "bytes_per_vector": 1536, + "threads": 1, + "ms_per_query": 1.64756916, + "gb_per_s": 7.491822680147764, + "recall_at_k": 1.0 + }, + { + "codec": "f32", + "bytes_per_vector": 1536, + "threads": 2, + "ms_per_query": 0.88228082, + "gb_per_s": 13.99021232264802, + "recall_at_k": 1.0 + }, + { + "codec": "f32", + "bytes_per_vector": 1536, + "threads": 4, + "ms_per_query": 0.50241584, + "gb_per_s": 24.567887827740464, + "recall_at_k": 1.0 + }, + { + "codec": "f32", + "bytes_per_vector": 1536, + "threads": 8, + "ms_per_query": 0.31641668, + "gb_per_s": 39.0096249034659, + "recall_at_k": 1.0 + }, + { + "codec": "f32", + "bytes_per_vector": 1536, + "threads": 10, + "ms_per_query": 0.28512666, + "gb_per_s": 43.29057128505627, + "recall_at_k": 1.0 + }, + { + "codec": "int8", + "bytes_per_vector": 392, + "threads": 1, + "ms_per_query": 2.1036358199999996, + "gb_per_s": 1.4974607154198394, + "recall_at_k": 0.984 + }, + { + "codec": "int8", + "bytes_per_vector": 392, + "threads": 2, + "ms_per_query": 1.1073724999999999, + "gb_per_s": 2.8446724114965836, + "recall_at_k": 0.984 + }, + { + "codec": "int8", + "bytes_per_vector": 392, + "threads": 4, + "ms_per_query": 0.61244416, + "gb_per_s": 5.143508920062199, + "recall_at_k": 0.984 + }, + { + "codec": "int8", + "bytes_per_vector": 392, + "threads": 8, + "ms_per_query": 0.36810834, + "gb_per_s": 8.557567590019831, + "recall_at_k": 0.984 + }, + { + "codec": "int8", + "bytes_per_vector": 392, + "threads": 10, + "ms_per_query": 0.32413832, + "gb_per_s": 9.718418976195101, + "recall_at_k": 0.984 + }, + { + "codec": "binary", + "bytes_per_vector": 48, + "threads": 1, + "ms_per_query": 1.9765516599999997, + "gb_per_s": 0.19515199516717924, + "recall_at_k": 0.63 + }, + { + "codec": "binary", + "bytes_per_vector": 48, + "threads": 2, + "ms_per_query": 1.02961916, + "gb_per_s": 0.37463172305379394, + "recall_at_k": 0.63 + }, + { + "codec": "binary", + "bytes_per_vector": 48, + "threads": 4, + "ms_per_query": 0.5693374999999999, + "gb_per_s": 0.6775032384130679, + "recall_at_k": 0.63 + }, + { + "codec": "binary", + "bytes_per_vector": 48, + "threads": 8, + "ms_per_query": 0.34715416, + "gb_per_s": 1.111114439763591, + "recall_at_k": 0.63 + }, + { + "codec": "binary", + "bytes_per_vector": 48, + "threads": 10, + "ms_per_query": 0.30904666000000003, + "gb_per_s": 1.248122209118843, + "recall_at_k": 0.63 + }, + { + "codec": "binary-centred", + "bytes_per_vector": 48, + "threads": 1, + "ms_per_query": 2.30894916, + "gb_per_s": 0.16705781430025077, + "recall_at_k": 0.66 + }, + { + "codec": "binary-centred", + "bytes_per_vector": 48, + "threads": 2, + "ms_per_query": 1.19929832, + "gb_per_s": 0.3216280666515067, + "recall_at_k": 0.66 + }, + { + "codec": "binary-centred", + "bytes_per_vector": 48, + "threads": 4, + "ms_per_query": 0.652355, + "gb_per_s": 0.59128541974845, + "recall_at_k": 0.66 + }, + { + "codec": "binary-centred", + "bytes_per_vector": 48, + "threads": 8, + "ms_per_query": 0.3913725, + "gb_per_s": 0.9855776785543184, + "recall_at_k": 0.66 + }, + { + "codec": "binary-centred", + "bytes_per_vector": 48, + "threads": 10, + "ms_per_query": 0.34174499999999997, + "gb_per_s": 1.1287012245972876, + "recall_at_k": 0.66 + }, + { + "codec": "binary-hamming", + "bytes_per_vector": 48, + "threads": 1, + "ms_per_query": 0.08798666000000001, + "gb_per_s": 4.3839372923122655, + "recall_at_k": 0.516 + }, + { + "codec": "binary-hamming", + "bytes_per_vector": 48, + "threads": 2, + "ms_per_query": 0.071535, + "gb_per_s": 5.392157685049276, + "recall_at_k": 0.516 + }, + { + "codec": "binary-hamming", + "bytes_per_vector": 48, + "threads": 4, + "ms_per_query": 0.0780025, + "gb_per_s": 4.945072273324573, + "recall_at_k": 0.516 + }, + { + "codec": "binary-hamming", + "bytes_per_vector": 48, + "threads": 8, + "ms_per_query": 0.09615831999999999, + "gb_per_s": 4.011384558299271, + "recall_at_k": 0.516 + }, + { + "codec": "binary-hamming", + "bytes_per_vector": 48, + "threads": 10, + "ms_per_query": 0.10813416000000001, + "gb_per_s": 3.567124394363446, + "recall_at_k": 0.516 + }, + { + "codec": "pq48", + "bytes_per_vector": 48, + "threads": 1, + "ms_per_query": 1.64010584, + "gb_per_s": 0.2351848219746599, + "recall_at_k": 0.732 + }, + { + "codec": "pq48", + "bytes_per_vector": 48, + "threads": 2, + "ms_per_query": 0.8723575, + "gb_per_s": 0.4421673453830568, + "recall_at_k": 0.732 + }, + { + "codec": "pq48", + "bytes_per_vector": 48, + "threads": 4, + "ms_per_query": 0.48843168, + "gb_per_s": 0.7897276442019485, + "recall_at_k": 0.732 + }, + { + "codec": "pq48", + "bytes_per_vector": 48, + "threads": 8, + "ms_per_query": 0.30167415999999997, + "gb_per_s": 1.2786245928388433, + "recall_at_k": 0.732 + }, + { + "codec": "pq48", + "bytes_per_vector": 48, + "threads": 10, + "ms_per_query": 0.277705, + "gb_per_s": 1.3889847139950666, + "recall_at_k": 0.732 + }, + { + "codec": "pq24", + "bytes_per_vector": 24, + "threads": 1, + "ms_per_query": 1.6194525000000002, + "gb_per_s": 0.11909210057102632, + "recall_at_k": 0.626 + }, + { + "codec": "pq24", + "bytes_per_vector": 24, + "threads": 2, + "ms_per_query": 0.85000668, + "gb_per_s": 0.226897040385612, + "recall_at_k": 0.626 + }, + { + "codec": "pq24", + "bytes_per_vector": 24, + "threads": 4, + "ms_per_query": 0.48382, + "gb_per_s": 0.3986275887726841, + "recall_at_k": 0.626 + }, + { + "codec": "pq24", + "bytes_per_vector": 24, + "threads": 8, + "ms_per_query": 0.3063725, + "gb_per_s": 0.6295081967213115, + "recall_at_k": 0.626 + }, + { + "codec": "pq24", + "bytes_per_vector": 24, + "threads": 10, + "ms_per_query": 0.28832, + "gb_per_s": 0.6689234184239733, + "recall_at_k": 0.626 + }, + { + "codec": "mixed48-pca-w", + "bytes_per_vector": 48, + "threads": 1, + "ms_per_query": 3.97714, + "gb_per_s": 0.096986276570601, + "recall_at_k": 0.724 + }, + { + "codec": "mixed48-pca-w", + "bytes_per_vector": 48, + "threads": 2, + "ms_per_query": 2.1327083399999998, + "gb_per_s": 0.18086298663791978, + "recall_at_k": 0.724 + }, + { + "codec": "mixed48-pca-w", + "bytes_per_vector": 48, + "threads": 4, + "ms_per_query": 1.1435, + "gb_per_s": 0.33732225623087014, + "recall_at_k": 0.724 + }, + { + "codec": "mixed48-pca-w", + "bytes_per_vector": 48, + "threads": 8, + "ms_per_query": 0.6587908400000001, + "gb_per_s": 0.5855090517044832, + "recall_at_k": 0.724 + }, + { + "codec": "mixed48-pca-w", + "bytes_per_vector": 48, + "threads": 10, + "ms_per_query": 0.57014834, + "gb_per_s": 0.6765397229780585, + "recall_at_k": 0.724 + } + ], + "bandwidth": [ + { + "threads": 1, + "gb_per_s": 83.92432457697323 + }, + { + "threads": 2, + "gb_per_s": 138.7046858836502 + }, + { + "threads": 4, + "gb_per_s": 208.71138790485213 + }, + { + "threads": 8, + "gb_per_s": 224.56368413149295 + }, + { + "threads": 10, + "gb_per_s": 238.91234125419533 + } + ] +} diff --git a/experiments/2026-11-zero-copy-scan/results/summary.txt b/experiments/2026-11-zero-copy-scan/results/summary.txt new file mode 100644 index 0000000..548b2cb --- /dev/null +++ b/experiments/2026-11-zero-copy-scan/results/summary.txt @@ -0,0 +1,112 @@ +== fiqa-bge-small: 71274 vectors × 384 dims, 50 queries, k=10 + storage path (f32) ms/query speed-up + sqlite rows, copied 74.70 1.0× + sqlite rows, borrowed 54.50 1.4× + sqlite rows, borrowed, mmap 27.40 2.7× + cached buffer (Store::search) 14.68 5.1× + building the buffer ms MB + sqlite (borrowed blobs, one copy) 46.7 109.5 + flat code file (one read) 8.6 109.5 + memory bandwidth: 1 thread(s) 78 GB/s, 2 thread(s) 142 GB/s, 4 thread(s) 148 GB/s, 8 thread(s) 209 GB/s, 10 thread(s) 238 GB/s + codec B/vec thr ms/query ns/vector GB/s % bw recall@10 + f32 1536 1 14.762 207.12 7.42 9.5% 1.0000 + f32 1536 2 7.556 106.01 14.49 10.2% 1.0000 + f32 1536 4 3.996 56.07 27.40 18.5% 1.0000 + f32 1536 8 2.092 29.36 52.32 25.1% 1.0000 + f32 1536 10 1.706 23.93 64.19 26.9% 1.0000 + int8 392 1 18.638 261.49 1.50 1.9% 0.9780 + int8 392 2 9.545 133.92 2.93 2.1% 0.9780 + int8 392 4 4.897 68.70 5.71 3.8% 0.9780 + int8 392 8 2.560 35.92 10.91 5.2% 0.9780 + int8 392 10 2.089 29.31 13.37 5.6% 0.9780 + binary 48 1 17.555 246.30 0.19 0.3% 0.5040 + binary 48 2 8.920 125.15 0.38 0.3% 0.5040 + binary 48 4 4.581 64.28 0.75 0.5% 0.5040 + binary 48 8 2.429 34.08 1.41 0.7% 0.5040 + binary 48 10 1.961 27.51 1.74 0.7% 0.5040 + binary-centred 48 1 20.476 287.28 0.17 0.2% 0.5620 + binary-centred 48 2 10.372 145.53 0.33 0.2% 0.5620 + binary-centred 48 4 5.341 74.93 0.64 0.4% 0.5620 + binary-centred 48 8 2.781 39.02 1.23 0.6% 0.5620 + binary-centred 48 10 2.297 32.23 1.49 0.6% 0.5620 + binary-hamming 48 1 0.718 10.08 4.76 6.1% 0.3720 + binary-hamming 48 2 0.415 5.83 8.24 5.8% 0.3720 + binary-hamming 48 4 0.249 3.49 13.74 9.3% 0.3720 + binary-hamming 48 8 0.180 2.52 19.06 9.1% 0.3720 + binary-hamming 48 10 0.168 2.36 20.32 8.5% 0.3720 + pq48 48 1 14.442 202.63 0.24 0.3% 0.6440 + pq48 48 2 7.364 103.32 0.46 0.3% 0.6440 + pq48 48 4 3.783 53.08 0.90 0.6% 0.6440 + pq48 48 8 1.998 28.03 1.71 0.8% 0.6440 + pq48 48 10 1.621 22.74 2.11 0.9% 0.6440 + pq24 24 1 14.056 197.21 0.12 0.2% 0.4220 + pq24 24 2 7.252 101.75 0.24 0.2% 0.4220 + pq24 24 4 3.789 53.16 0.45 0.3% 0.4220 + pq24 24 8 1.986 27.87 0.86 0.4% 0.4220 + pq24 24 10 1.620 22.73 1.06 0.4% 0.4220 + mixed48-pca-w 48 1 32.935 462.09 0.10 0.1% 0.7080 + mixed48-pca-w 48 2 17.342 243.31 0.20 0.1% 0.7080 + mixed48-pca-w 48 4 8.489 119.11 0.40 0.3% 0.7080 + mixed48-pca-w 48 8 4.596 64.48 0.74 0.4% 0.7080 + mixed48-pca-w 48 10 3.840 53.88 0.89 0.4% 0.7080 + hybrid estimate, 1 thread: scan + N f32 re-scores at 207.1 ns each (random access ignored) + binary-hamming scan 0.718 ms → N=50: 0.729 ms, N=100: 0.739 ms, N=200: 0.760 ms (exact f32 scan 14.762 ms) + binary scan 17.555 ms → N=50: 17.565 ms, N=100: 17.576 ms, N=200: 17.597 ms (exact f32 scan 14.762 ms) + pq48 scan 14.442 ms → N=50: 14.453 ms, N=100: 14.463 ms, N=200: 14.484 ms (exact f32 scan 14.762 ms) + +== scifact-bge-small: 8036 vectors × 384 dims, 50 queries, k=10 + storage path (f32) ms/query speed-up + sqlite rows, copied 8.75 1.0× + sqlite rows, borrowed 6.41 1.4× + sqlite rows, borrowed, mmap 3.01 2.9× + cached buffer (Store::search) 1.68 5.2× + building the buffer ms MB + sqlite (borrowed blobs, one copy) 6.6 12.3 + flat code file (one read) 1.8 12.3 + memory bandwidth: 1 thread(s) 84 GB/s, 2 thread(s) 139 GB/s, 4 thread(s) 209 GB/s, 8 thread(s) 225 GB/s, 10 thread(s) 239 GB/s + codec B/vec thr ms/query ns/vector GB/s % bw recall@10 + f32 1536 1 1.648 205.02 7.49 8.9% 1.0000 + f32 1536 2 0.882 109.79 13.99 10.1% 1.0000 + f32 1536 4 0.502 62.52 24.57 11.8% 1.0000 + f32 1536 8 0.316 39.37 39.01 17.4% 1.0000 + f32 1536 10 0.285 35.48 43.29 18.1% 1.0000 + int8 392 1 2.104 261.78 1.50 1.8% 0.9840 + int8 392 2 1.107 137.80 2.84 2.1% 0.9840 + int8 392 4 0.612 76.21 5.14 2.5% 0.9840 + int8 392 8 0.368 45.81 8.56 3.8% 0.9840 + int8 392 10 0.324 40.34 9.72 4.1% 0.9840 + binary 48 1 1.977 245.96 0.20 0.2% 0.6300 + binary 48 2 1.030 128.13 0.37 0.3% 0.6300 + binary 48 4 0.569 70.85 0.68 0.3% 0.6300 + binary 48 8 0.347 43.20 1.11 0.5% 0.6300 + binary 48 10 0.309 38.46 1.25 0.5% 0.6300 + binary-centred 48 1 2.309 287.33 0.17 0.2% 0.6600 + binary-centred 48 2 1.199 149.24 0.32 0.2% 0.6600 + binary-centred 48 4 0.652 81.18 0.59 0.3% 0.6600 + binary-centred 48 8 0.391 48.70 0.99 0.4% 0.6600 + binary-centred 48 10 0.342 42.53 1.13 0.5% 0.6600 + binary-hamming 48 1 0.088 10.95 4.38 5.2% 0.5160 + binary-hamming 48 2 0.072 8.90 5.39 3.9% 0.5160 + binary-hamming 48 4 0.078 9.71 4.95 2.4% 0.5160 + binary-hamming 48 8 0.096 11.97 4.01 1.8% 0.5160 + binary-hamming 48 10 0.108 13.46 3.57 1.5% 0.5160 + pq48 48 1 1.640 204.09 0.24 0.3% 0.7320 + pq48 48 2 0.872 108.56 0.44 0.3% 0.7320 + pq48 48 4 0.488 60.78 0.79 0.4% 0.7320 + pq48 48 8 0.302 37.54 1.28 0.6% 0.7320 + pq48 48 10 0.278 34.56 1.39 0.6% 0.7320 + pq24 24 1 1.619 201.52 0.12 0.1% 0.6260 + pq24 24 2 0.850 105.77 0.23 0.2% 0.6260 + pq24 24 4 0.484 60.21 0.40 0.2% 0.6260 + pq24 24 8 0.306 38.12 0.63 0.3% 0.6260 + pq24 24 10 0.288 35.88 0.67 0.3% 0.6260 + mixed48-pca-w 48 1 3.977 494.92 0.10 0.1% 0.7240 + mixed48-pca-w 48 2 2.133 265.39 0.18 0.1% 0.7240 + mixed48-pca-w 48 4 1.143 142.30 0.34 0.2% 0.7240 + mixed48-pca-w 48 8 0.659 81.98 0.59 0.3% 0.7240 + mixed48-pca-w 48 10 0.570 70.95 0.68 0.3% 0.7240 + hybrid estimate, 1 thread: scan + N f32 re-scores at 205.0 ns each (random access ignored) + binary-hamming scan 0.088 ms → N=50: 0.098 ms, N=100: 0.108 ms, N=200: 0.129 ms (exact f32 scan 1.648 ms) + binary scan 1.977 ms → N=50: 1.987 ms, N=100: 1.997 ms, N=200: 2.018 ms (exact f32 scan 1.648 ms) + pq48 scan 1.640 ms → N=50: 1.650 ms, N=100: 1.661 ms, N=200: 1.681 ms (exact f32 scan 1.648 ms) + diff --git a/experiments/2026-11-zero-copy-scan/run.sh b/experiments/2026-11-zero-copy-scan/run.sh new file mode 100755 index 0000000..8fb1e59 --- /dev/null +++ b/experiments/2026-11-zero-copy-scan/run.sh @@ -0,0 +1,35 @@ +#!/usr/bin/env bash +# Zero-copy scan: where a brute-force search spends its time, and what each codec's scan costs +# against the machine's memory bandwidth. Times the storage paths (SQLite rows copied, SQLite +# rows borrowed with and without SQLite's memory map, the cached contiguous buffer), the cost of +# building the buffer, and every codec scanned in place at several thread counts, with recall@10 +# against exact f32. Run on a quiet machine: nothing else should be using the CPU. +# Reuses ../2026-11-neighbour-spacing's collections. Needs: a built `hvec` (cargo build --release). +set -euo pipefail +HERE="$(cd "$(dirname "$0")" && pwd)" +ROOT="$HERE/../.." +HVEC="${HVEC:-$ROOT/target/release/hvec}" +SPACING="$HERE/../2026-11-neighbour-spacing" +export HVEC_CONFIG="${HVEC_CONFIG:-$SPACING/work/config.toml}" +COLLECTIONS="${COLLECTIONS:-scifact-bge-small fiqa-bge-small}" +CODECS="${CODECS:-f32,int8,binary,binary-centred,binary-hamming,pq48,pq24,mixed48-pca-w}" +THREADS="${THREADS:-1,2,4,8,10}" +SAMPLE="${SAMPLE:-50}" +REPEAT="${REPEAT:-3}" +mkdir -p "$HERE/results" +[ -f "$HVEC_CONFIG" ] || { echo "build the collections first: $SPACING/run.sh"; exit 1; } + +{ + echo "hvec $(git -C "$ROOT" rev-parse --short HEAD)$(git -C "$ROOT" diff --quiet || echo '+dirty')" + sysctl -n machdep.cpu.brand_string 2>/dev/null || true + sysctl -n hw.perflevel0.physicalcpu hw.perflevel1.physicalcpu 2>/dev/null | paste -sd' ' - | sed 's/^/performance and efficiency cores: /' || true + sysctl -n hw.memsize 2>/dev/null | awk '{printf "memory: %.0f GB\n", $1/1e9}' || true +} > "$HERE/results/machine.txt" + +for col in $COLLECTIONS; do + "$HVEC" bench scan --collection "$col" --codecs "$CODECS" --threads "$THREADS" --sample "$SAMPLE" \ + --repeat "$REPEAT" --json > "$HERE/results/$col.json" + echo "== $col done" +done +python3 -I "$HERE/summarise.py" "$HERE/results" | tee "$HERE/results/summary.txt" +cat "$HERE/results/machine.txt" diff --git a/experiments/2026-11-zero-copy-scan/summarise.py b/experiments/2026-11-zero-copy-scan/summarise.py new file mode 100644 index 0000000..27a2432 --- /dev/null +++ b/experiments/2026-11-zero-copy-scan/summarise.py @@ -0,0 +1,51 @@ +"""Summarise `bench scan --json` reports. + + python3 -I summarise.py + +For each collection: the storage paths with their speed-up over the original row scan, the cost +of building the buffer, and each codec's scan (ms/query, ns per vector, GB/s, share of the measured +memory bandwidth, recall@10). Then, per collection, the hybrid estimate: a scan plus N f32 +re-scores, using the measured per-vector costs. +""" + +import json +import sys +from pathlib import Path + + +def main(results: Path) -> None: + for path in sorted(results.glob("*.json")): + r = json.loads(path.read_text()) + n = r["vectors"] + print(f"== {r['collection']}: {n} vectors × {r['dimension']} dims, {r['queries']} queries, k={r['k']}") + base = next((p["ms_per_query"] for p in r["paths"] if p["path"].startswith("sqlite rows, copied")), None) + print(f" {'storage path (f32)':<34} {'ms/query':>9} {'speed-up':>9}") + for p in r["paths"]: + up = f"{base / p['ms_per_query']:.1f}×" if base else "-" + print(f" {p['path']:<34} {p['ms_per_query']:>9.2f} {up:>9}") + print(f" {'building the buffer':<34} {'ms':>9} {'MB':>9}") + for l in r["loads"]: + print(f" {l['source']:<34} {l['ms']:>9.1f} {l['bytes'] / 1e6:>9.1f}") + bw = {b["threads"]: b["gb_per_s"] for b in r["bandwidth"]} + print(f" memory bandwidth: " + ", ".join(f"{t} thread(s) {g:.0f} GB/s" for t, g in sorted(bw.items()))) + print(f" {'codec':<16} {'B/vec':>6} {'thr':>4} {'ms/query':>9} {'ns/vector':>10} {'GB/s':>7} {'% bw':>6} {'recall@10':>10}") + for c in r["codecs"]: + ns = c["ms_per_query"] * 1e6 / n + share = 100 * c["gb_per_s"] / bw[c["threads"]] if c["threads"] in bw else float("nan") + print(f" {c['codec']:<16} {c['bytes_per_vector']:>6} {c['threads']:>4} {c['ms_per_query']:>9.3f} {ns:>10.2f} " + f"{c['gb_per_s']:>7.2f} {share:>5.1f}% {c['recall_at_k']:>10.4f}") + one = {c["codec"]: c for c in r["codecs"] if c["threads"] == 1} + if "f32" in one: + f32_ns = one["f32"]["ms_per_query"] * 1e6 / n + print(f" hybrid estimate, 1 thread: scan + N f32 re-scores at {f32_ns:.1f} ns each (random access ignored)") + for scan in ("binary-hamming", "binary", "pq48"): + if scan not in one: + continue + s_ms = one[scan]["ms_per_query"] + cells = ", ".join(f"N={d}: {s_ms + d * f32_ns / 1e6:.3f} ms" for d in (50, 100, 200)) + print(f" {scan:<16} scan {s_ms:.3f} ms → {cells} (exact f32 scan {one['f32']['ms_per_query']:.3f} ms)") + print() + + +if __name__ == "__main__": + main(Path(sys.argv[1]))