Repository navigation
Views: base references, reclaim and base-builder admission - #997
Open
aft-alfonso[bot] wants to merge 8 commits into
Open
aft-alfonso[bot] wants to merge 8 commits into
aft-alfonso[bot] wants to merge 8 commits into
Conversation
This was referenced Oct 26, 2026
…robe Adds the base_refs, base_builds, strict_rebuild, base_sweeps and sweep_state tables as registry format/schema 2/2 (reserved for slice 4 in docs/design/views-v3-format.md), installed only by an exact-predecessor upgrade from 1/1 after the registry reader floor is raised to 2. Registries created by the 3a copy path stay 1/1. views_v3_registry is now advertised as readable at 2, so reader_floor's advertisement test and the v3 newer-format refusal test (which used 2 as 'newer') move to 2 and 3 respectively. Also: barrier-only registry opens (no covering measurement), a plane-row sweep over the family's v3 plane stores, LedgerReader member/read helpers, and GetDiskFreeSpaceExW for the non-unix free-space probe, which 3a left unsupported. Co-authored-by: alfonso-aft <289616620+alfonso-aft@users.noreply.github.com>
Base identity: tagged, length-prefixed BLAKE3 digests of a base manifest's membership and content (readiness excluded; head_tree and git_oid refused, slice 4 bases are membership-keyed) and of its effective identity (family, producers, ignore fingerprint, platform family, membership). The base scope name derives from the identity digest. resolve_base pins the base's current generation with a read marker, re-verifies pointer and member row, and recomputes both digests from the pinned manifest; only that AuthenticatedBase validates a BaseHeader (bound_generation stays diagnostic). Worker side: publish_worker runs the insertion barrier (BaseRefMarker with a per-acquisition nonce plus an intent row in one IMMEDIATE transaction under the reservation lock), authenticates the base after the barrier, rebinds the delta to the authenticated header, verify_held and a base re-check right before the pointer CAS, then ReferrerPublished. Conflict/EquivalentWinner and failed verifies delete only their own intent row under the barrier before the marker drops; a CAS that may have landed is published, never cleaned up. open_foreign_v3 (and read::open_foreign_v3_generation) does the dual pin-then-verify with three attempts and owns both markers. read::open_foreign_generation keeps refusing a base header, now because a v2 view can never hold an authenticatable base reference (rule 6); its test asserts the new reason. Sweep: marking of current, read-marked and pinned generations under the reservation lock, exact edges, dangling edges counted and recorded in strict_rebuild without aborting, bounded reconciliation (256 rows / 25 ms, updated_epoch < S only, wrap-around position), repair of valid edges, Hold::Referenced via eviction::v3_base_hold, the two-sweep reclaim, and the plane-row sweep. The retention collector runs it per v3 family visit and deletes a deregistered worker's rows with its member row. DurabilityStep gains the seven base-path boundaries (BASE_PATH); the v2 kill test's step list excludes them. Tests use injected EIO/EXDEV/ENOSPC at each boundary through contracts::inject_fault_at. Co-authored-by: alfonso-aft <289616620+alfonso-aft@users.noreply.github.com>
base_builds claim with the owner-token CAS: insert when absent, Busy while the owner lives (running, stopped or uncertain), takeover of a dead owner with attempt+1 and a fresh token, after which the dead attempt's staging/a<attempt>-<token>-* directories are removed. Staging and derived names carry (attempt, owner_token). The live-base cap (64 per family, counting claimed-but-unregistered bases) refuses new claims and counts the hit in sweep_state.live_base_cap_hits. plan_binder: only the claim owner builds; others wait (ACK + direct scan) and go strict only on a failed claim, the cap, or the ruled 600 s wait. initial_reservation implements the ruled formula with the 1 GiB cap. DiskBudget::admit_v3/extend_v3 admit v3 reservations against the family hard limit (refused as storage_gap:v3_family_hard_limit, no legacy fallback) and flag usage over the 2 GiB target; RegistryWriter::extend grows a live reservation under the floor and namespace checks. No limit or budget constant changes. Co-authored-by: alfonso-aft <289616620+alfonso-aft@users.noreply.github.com>
build_base reads the walker's membership with stability-checked reads (equal size, mtime and device/inode before and after, three attempts), builds a membership-keyed manifest (head_tree None, no git_oid), claims the base, admits the ruled initial reservation through DiskBudget::admit_v3 (a refusal releases the claim and returns the storage gap for the strict path), registers the base, writes the lexical plane into the v3 trigram store (trigram::materialize_v3, keys protected by a v3 live pin first) and first-publishes with graph and semantic planes pending. publish_successor lets only the live claim owner publish monotone readiness successors; a model outage is recorded as Failed while lexical serving continues. v3 generations carry no trigram segment: v3 has no segment rows or segment collection yet, so readers build from payloads. Co-authored-by: alfonso-aft <289616620+alfonso-aft@users.noreply.github.com>
- G7: two child binders (one SIGSTOPped with its intent row across two sweeps, then killed) plus in-process binds with one aborting; the base is never counted, the stopped owner's intent is kept, then orphaned, then retired, and published references keep the base. - Kill test: a child parks at each DurabilityStep::BASE_PATH boundary (the child refuses unknown or non-base step names; the parent checks the ready file names the step), is SIGKILLed, and a fresh process sweeps: dead reservations are released, every published reference resolves, a dead BaseClaimed claim is taken over with attempt 2, and nothing appears under <storage>/views/v3. - Delete main, kill main's daemon, run GC in a standalone process, restart the worker: it composes over its base with correct contents. Co-authored-by: alfonso-aft <289616620+alfonso-aft@users.noreply.github.com>
… format doc Co-authored-by: alfonso-aft <289616620+alfonso-aft@users.noreply.github.com>
…m the republication row test The republication test now runs its sweep with a 60 s reconciliation time budget: under a loaded test runner the ruled 25 ms budget could end the pass before its final empty batch, which marks the pass incomplete without changing the rows the test asserts. Co-authored-by: alfonso-aft <289616620+alfonso-aft@users.noreply.github.com>
Co-authored-by: alfonso-aft <289616620+alfonso-aft@users.noreply.github.com>
ualtinok
force-pushed
the
land/views-c-base-references
branch
from
October 27, 2026 06:01
142f103 to
8fd657c
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #993 (views slice 4-2); retarget to main after #993 merges. Views stay off by default; no routing change.
Deferred to the views measurement step: the previous-release sweep over v3 data, the native macOS gate script, native Windows runs.