Repository navigation
Conversation
|
| Filename | Overview |
|---|---|
| archivist/blockexchange/engine/discovery.nim | Adds bounded per-CID discovery backoff and removes stale backoff state when blocks leave the want list. |
| archivist/blockexchange/engine/engine.nim | Rotates refresh targets through a query quarantine while allowing newly queued wants to bypass it. |
| archivist/blockexchange/peers/peercontext.nim | Tracks query timing and gives unscored peers a cold-start aggregate score. |
| archivist/marketplace/purchasing.nim | Re-adopts signer-owned purchases from historical storage-request events during startup. |
| archivist/marketplace/sales.nim | Re-adopts signer-owned sales slots from historical slot-filled events during startup. |
| archivist/streams/storestream.nim | Retries missing indices from partial block batches before ordered stream copying. |
| archivist/archivist.nim | Retries marketplace connection attempts during persistent node startup. |
| archivist/rest/api.nim | Avoids completing responses whose streamed body is shorter than the declared content length. |
Flowchart
%%{init: {'theme': 'neutral'}}%%
flowchart TD
A[Node startup] --> B{Marketplace reachable?}
B -->|No| C[Retry with configured delay]
C --> B
B -->|Yes| D[Load historical marketplace events]
D --> E[Re-adopt owned purchases and sales]
E --> F[Start block exchange]
F --> G{Enough peers or providers?}
G -->|No| H[Back off discovery and rotate queried peers]
H --> G
G -->|Yes| I[Fetch and stream blocks]
I --> J{Batch complete?}
J -->|No| K[Retry missing indices]
K --> J
J -->|Yes| L[Complete response]
Reviews (2): Last reviewed commit: "Pin glibc mmap threshold below block siz..." | Re-trigger Greptile
- Retry marketplace connect and cirdl circuit download (10x10s) - Abort REST retrieves with short bodies loudly instead of silently truncating
The knowledge-refresh fallback always asked the same aggregate top-K; stable scores and deterministic tie order pinned the same peers forever, so a fresh DHT-discovered provider was never refreshed, never reported presence, and was never fetched from. Add a query quarantine that skips peers recently sent a want list, randomize equal-score ordering in the rankers, and exempt peers with unsent wants so newly queued blocks still reach them. Reuse the orphaned topPeersForCid helper for the known-haves branch and wake at the earliest quarantine expiry instead of busy-looping.
- Remove the 10-minute advertise retry window: the sparse retry is now steady-state, gated only by table health, so small networks keep replicating provider records to newly-arriving peers - Rework the discovery find-backoff to a (due, mult) tuple table with multiplier-derived intervals; prune entries once a cid leaves the want list, including tree cids of leaf wants - Make nodesDiscovered a method so tests can override it; export DefaultMinAdvertisePeers - Add steady-state advertiser retry test and backoff-recovery test
erasure.repair stored the reconstructed dataset under the original-tree overlay with no expiry, so putOverlay assigned the 30-day default TTL and the maintenance never cleaned it after the request ended (hosts retained the whole dataset - SlotCleanupTest 97 vs 33 blocks). Thread the request expiry from storeSlot through repair into the overlay writes, matching the flow's implicit cleanup design.
- Clamp pastBlockTag to genesis when the requested window exceeds the chain head: a fresh chain is shorter than the 30-day request window, and the UInt256 subtraction underflowed, failing every marketplace start with 'hex number > 64 bits' - Reload purchases and sales from onchain StorageRequested/SlotFilled event history instead of myRequests/mySlots - Guard errored-state restarts with the onchain request/slot state and signer check instead of the myRequests/mySlots membership list - Reserve slots only when the host token balance covers the collateral - Pin vendored contractabi with the readTail in-place decode fallback
The event-based sales reload created sale agents for every SlotFilled event in the window, including other hosts' fills, so the sales slots list could return foreign slots that 404 on lookup and break clients asking for them. Look up each slot's onchain host and skip slots that are not ours.
The onchain reservation allows several hosts per slot with first-fill-wins, so a host with three queue workers could reserve most slots of a request and then race other hosts on the same slot. Skip queue items for a request that already has an agent.
This reverts commit ba1d2d4.
128 KiB blocks (from 512 KiB) plus smaller store/serve batches cut the in-flight peaks that glibc's arena retention pins as RSS: upload 2 GiB -> 512 MiB, serve 72 MiB/peer -> 6 MiB/peer. Old datasets still decode via manifest blockSize.
A single transient fetch miss (peer dropped a want-batch) or a store CAS retry-out aborted the whole download stream. Re-fetch missing indices up to 5 times with backoff before failing.
Concurrent putBlocks on the same treeCid (swarm downloads from many peers) CAS-conflict on the shared overlay/refCount records; the 10-retry bound could exhaust under the burst. 30 retries let the writer storm drain; the middleware refetches and re-applies on fresh values.
glibc's mmap-threshold ratchet demotes the 128 KiB block buffers to the heap, where interleaved small allocations pin the arena tops and freed chunks never return to the OS - RSS stays at the historical peak forever (measured: 627 MB RSS -> 61 MB after malloc_trim(0)). Pinning the threshold to 64 KiB keeps block buffers on the mmap path so frees return memory at once, and disables the ratchet. Makes the 20-node x 300 MB StreamOneToMany swarm leg pass with flat memory (nodes ~700 MB vs 2.5 GB hoarded); threshold beats per-poll malloc_trim in the A/B (no measurable speed cost, trim was 7-10% slower at swarm scale).
dryajov
force-pushed
the
fix/battery-stability
branch
from
August 19, 2026 15:07
5d15143 to
ea10326
Compare
There was a problem hiding this comment.
Your organization has used all 50 credits included in the free plan this billing period. To keep receiving reviews, upgrade your plan.
The branch carried an old contractabi gitlink (1c41c22e) that was never pushed to the durability-labs fork - CI submodule fetch fails with 'not our ref'. Main's pin (e9092906, v0.7.4) is reachable and the branch was rebased on it; align the pin and the CI submodule stage passes.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.