[MOD-18624] Cover tiered get_vector and read uint8 vectors back - #1049
Merged
Merged
Conversation
The Python get_vector() dispatcher used to cast every non brute force index to HNSWIndex, so a tiered index failed the cast and the null result was dereferenced. That cast was replaced in #1045, which leaves the behaviour uncovered by a test: the tiered reads in the flow tests all run after ingestion has drained, so nothing exercises a read while the vector is still in the flat buffer. Add tiered get_vector tests for the single and multi cases that read a label back both before and after ingestion drains, and assert an absent label is reported as no vectors. The multi case asserts a count range before draining, because an ingest job inserts into the backend before removing from the buffer and a vector caught inside that window is reported by both tiers. getVector() also had no branch for uint8, so reading a uint8 vector back raised "Invalid vector data type" even though the type is otherwise supported. Add the branch, and a read back test for int8 and uint8. SQ8 tiered coverage is not included: quantType is not exposed to Python on main, so those tests belong with the bindings that expose it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
dor-forer
marked this pull request as ready for review
September 15, 2026 07:13
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #1049 +/- ##
==========================================
- Coverage 97.50% 97.49% -0.02%
==========================================
Files 141 141
Lines 8851 8851
==========================================
- Hits 8630 8629 -1
- Misses 221 222 +1 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
nonirosenfeldredis
approved these changes
Sep 15, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Two independent gaps around the Python
get_vector(), both on uncompressed indexes.1. Tiered
get_vector()has no test coverage. The dispatcher used to cast every non brute force index toHNSWIndex, so a tiered index failed the cast and the null result was dereferenced (SIGSEGV). #1045 replaced that cast with a tiered-first one, which fixed the crash but left the behavior untested: every tiered read in the flow tests runs afterwait_for_index(), so nothing exercises a read while the vector is still in the flat buffer.This adds
test_get_vectorandtest_get_vector_multitotests/flow/test_hnsw_tiered.py. Both read a label back before and after ingestion drains, and assert an absent label is reported as no vectors. The multi case asserts a count range before draining: an ingest job inserts into the backend before removing from the buffer, so a vector caught inside that window is reported by both tiers.2.
getVector()had nouint8branch. Reading a uint8 vector back raisedRuntimeError: Invalid vector data type, even though uint8 is supported everywhere else in the bindings (knn_query,range_query, index construction). Adds the branch, plus a read-back test for int8 and uint8 intests/flow/test_hnsw.py.Verification
Built the Python bindings and ran the flow tests on a Linux x86_64 box:
TestUINT8::test_get_vectorfails withRuntimeError: Invalid vector data type;TestINT8::test_get_vectorpasses.tests/flow/test_hnsw_tiered.py: 21 passed.get_vectorafteradd_vector+wait_for_index) returns(1, 64)with the inserted values, no SIGSEGV.check-format.shclean.Not included
SQ8 tiered coverage.
HNSWParams::quantTypeis not exposed to Python on main, so the SQ8 half of MOD-18624 belongs with #1048, which adds that binding. MOD-18624 stays open until that test lands there.🤖 Generated with Claude Code
Note
Low Risk
Small binding branch plus test-only coverage; no auth or core query-path logic changes beyond aligning UINT8 with existing INT8 read-back behavior.
Overview
Fixes Python
get_vector()for UINT8 indexes and adds flow tests forget_vectoron plain HNSW and tiered HNSW.In
bindings.cpp,getVectornow handlesVecSimType_UINT8(same pattern as INT8:uint8_tstorage returned as float numpy), so UINT8 read-back no longer raises Invalid vector data type.test_hnsw.pyadds a sharedget_vectorcheck on the cached L2 index (exact round-trip for int types, empty(0, dim)for missing labels) and wirestest_get_vectorintoTestINT8andTestUINT8.test_hnsw_tiered.pyaddstest_get_vectorandtest_get_vector_multi, reading labels beforewait_for_index()(buffer vs backend) and after drain; the multi test allows up to2 * per_labelrows mid-ingest when both tiers briefly report the same vector.Reviewed by Cursor Bugbot for commit 2c7378c. Bugbot is set up for automated code reviews on this repo. Configure here.