fix: Count validity bits only up to the column length - #222
Open
SamuelSchlesinger wants to merge 2 commits into
Open
fix: Count validity bits only up to the column length#222SamuelSchlesinger wants to merge 2 commits into
SamuelSchlesinger wants to merge 2 commits into
Conversation
bitmapSlice's byte-aligned path returns whole bytes, so the last byte of a slice still carries validity bits for rows past the end of the slice. numElements and allMissing fold popCount over the whole byte vector and count them, so summarize reports Count 8 for a three-row take and for a five-row range, and an all-null slice no longer reads as all-null.
numElements and allMissing folded over the whole byte vector, so they counted the trailing bits a byte-aligned bitmapSlice leaves describing rows outside the slice. Add popCountUpTo and pass the column length, which keeps the aligned slice O(1) rather than copying to mask the last byte.
SamuelSchlesinger
force-pushed
the
fix/bitmap-slice-padding
branch
from
August 24, 2026 12:36
818cf21 to
1b40494
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
bitmapSlicekeeps whole bytes on its aligned path, so the trailing bits of a sliced bitmap still describe rows outside the slice.numElementsandallMissingfolded the whole byte vector, sosummarizeaftertake/rangeover-counted non-null rows and an all-null slice could read as not-all-null.Add
popCountUpToto the Bitmap module and stop at the column length.