Repository navigation
Price what the container kinds block refs cost where they live - #2554
Merged
Merged
Conversation
`what_folding_the_block_refs_into_the_slot_would_save` priced the slot fold BEFORE it
was built, and the fold landed at the figure it predicted. This is the same step for
the kinds that fold did not reach: hash, set, zset and list, whose rows live in
`RecordedMap` and whose refs are therefore still served from `object_block_lookup`.
It is wanted because nobody had measured what that lookup costs on a corpus that has
container objects. The published per-key figure is measured on a string-only corpus,
where the lookup reads 0.0 B/key for the honest reason that every string had its refs
adopted into its slot row. So the value of retiring the lookup was unknown, and a
saving that is unknown cannot be weighed against what it costs.
MEASURED, resident:
1792 objects: 139272 B 77.72 B/object
7168 objects: 560136 B 78.14 B/object
slope between the arms: 78.29 B/object
The slope and the per-object figure agree to 0.2%, which is the phase discipline the
sibling arm uses and the reason both counts are 7/8 of a power of two: a B-tree or
hashbrown cost per key is a step function of n, so two arms in different phases can
differ with nothing to give it away.
THE TRADE, BOTH SIDES. `object_block_lookup` is `#[serde(default, skip_serializing)]`
-- DERIVED, rebuilt from what the decode reads -- so every byte it holds is RESIDENT
and none is durable. The row it would move into is in a DURABLE map. So this is not
"two structures become one": it is resident bytes removed against durable bytes added,
at most 78.14 removed against at most 24.00 added, a net 54.14 B/object resident.
WHAT IT DOES NOT CLAIM, and this is the part worth reading twice:
* It does NOT move the published figure. That arm's corpus is string-only, where the
lookup is already 0.0 B/key, so retiring it changes nothing there and this is not
progress against the 25.0 B/key the asymptote still needs.
* A CEILING on the saving. The lookup is nested, model then key, and what a flat
per-object home would cost is not priced here.
* The paid side is an UPPER BOUND, not a measurement: a `BlockRefs` moved into an
existing value costs at most its own width and could cost nothing if it lands in
padding that value already carries.
* It prices the resident side only, and takes no position on whether the move should
happen -- `ObjectBlockLookup`'s own doc declines it on a dump-manifest-digest
ground that this arm does not touch.
`#[ignore]` and `alloc-probe` gated, as the sibling arm is, because it builds nested
maps at two corpus sizes under the counting allocator. No production code, no stored
format, no stamp.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why this exists before the migration, not after
what_folding_the_block_refs_into_the_slot_would_savepriced the slot fold before it wasbuilt, and the fold then landed at the figure that arm predicted. This is the same step for the
kinds that fold did not reach —
hash,set,zset,list— whose rows live inRecordedMapand whose refs are therefore still served from
object_block_lookup.Nobody had measured what that lookup costs on a corpus with container objects. The published
per-key figure uses a string-only corpus, where the lookup reads 0.0 B/key for the honest
reason that every string had its refs adopted into its slot row. So the value of retiring the
lookup was unknown — and a saving that is unknown cannot be weighed against what it costs.
Measured, resident
The slope and the per-object figure agree to 0.2%. That is the phase discipline the sibling
arm uses, and the reason both counts are 7/8 of a power of two: a B-tree or hashbrown cost per key
is a step function of n, so two arms in different phases can differ with nothing to give it away.
The arm asserts that agreement rather than printing it.
The trade, both sides
object_block_lookupis#[serde(default, skip_serializing)]— derived, rebuilt from what thedecode reads — so every byte it holds is resident and none is durable. The row it would move into
is in a durable map. So this is not "two structures become one":
What it does not claim
already 0.0 B/key, so retiring it changes nothing there. This is not progress against the
25.0 B/key the asymptote still needs.
home would cost is not priced here.
BlockRefsmoved into an existingvalue costs at most its own width, and could cost nothing if it lands in padding that value
already carries.
ObjectBlockLookup's own doc declines it on a dump-manifest-digest ground this arm does nottouch.
Shape
#[ignore]andalloc-probegated, as the sibling arm is, because it builds nested maps at twocorpus sizes under the counting allocator. No production code, no stored format, no stamp.
Verified on the merged tree:
--all-targetsand--all-targets --features alloc-probeboth at 0errors, and the arm passes with both of its assertions live (slope-versus-per-object agreement, and
a positive saving).
🤖 Generated with Claude Code