Repository navigation
ensemble: route prompts to workers found by gossip - #64
Merged
Merged
Conversation
A prompt naming a model no local bee advertises got an error, even when a peer humd advertised one over hum/hives/announce. hive_discover now has an unfiltered form; humd keeps what it hears keyed by the humd that can route to it, and on a local worker miss picks a connected peer advertising that model and routes the prompt with to/from set. The far side already resolved remote prompts by model and replied by sid via sid_origins, so one hop was the whole gap. A no-worker error now also goes back to a remote caller instead of dying in the local session. Re-advertise on PeerAdd: a bee only advertises when it handshakes with its own humd, which a peer reconnect does not re-trigger. PeerRemove drops that humd's manifests, and selection ignores any humd absent from the peer set, so a manifest cannot outlive its peer. A failed forward falls through to the error reply rather than returning, so a dead peer cannot hang the caller.
Adds prose for sim/tests/remote_discovery.rs and records which tests have no scenario yet, so the 1:1 claim in this README stops implying coverage that does not exist.
eviction_only_touches_the_dead_peer waited for Liveness::Dead, which is only set once a background drain task observes the closed transport. That is a scheduling assumption, not a guarantee, so the test timed out on a loaded CI runner while passing locally every time. A peer nobody is talking to stops being Live once the TTL passes, which is wall-clock guaranteed. Poll the sweep for the reap instead, and probe the surviving peer each pass so it is provably still fresh at reap time. With a 120ms TTL the old assertion only held because it ran within ~40ms of the probe; any correct fix to the wait would have let the live peer expire too and tripped the next assertion. 5.07s -> 0.40s, and no longer dependent on task scheduling.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #63.
What
A prompt naming a model that no local bee advertises got
chi:"error", even when a peer humd advertised a worker for it overhum/hives/announce. Now it routes there.The far side already worked. humd parses
fromon ensemble-sourced prompts, recordssid_origins, and routes worker replies back to the origin — so the missing piece was exactly one hop.How
Ensemble::hive_discover_all()— unfiltered manifests.hive_discover(name)is now a filter over it.to/from, route.PeerAdd. A bee advertises only when it handshakes with its own humd, which a peer reconnect does not re-trigger — so reconnect left peers invisible.PeerRemovedrops that humd's manifests; selection ignores any humd absent fromens.peers(), so a manifest cannot outlive its peer.Proof
sim/tests/remote_discovery.rs— two humds. The laptop has no worker and sends a barechi:"prompt"naming only a model; the server's worker is the sole advertiser and the laptop has never heard of it.Fails without the fix:
Passes with it. Full suite 210 passed / 0 failed; clippy clean under the CI command.
Notes
ens.peers()-gated rather than lease-gated. A manifest has no timestamp and there is no re-advertise heartbeat, so TTL eviction would drop live entries. The peer set is the honest liveness signal, and it is free.pick_overflow_peerstill selects by caps/free_slotsfor overflow — deliberately untouched, see below.Not in scope
pick_overflow_peerand manifest-discovery are now two selection mechanisms that can drift. Worth unifying, but that is a behaviour change to overflow routing and deserves its own issue.Address-level discovery is also still hand-rolled:
IrohTransport::connectrequires aniroh:hint and passes no relay URL, so an EndpointId alone cannot dial us. iroh 1.0'sendpoint_info(Pkarr + DNS, withUserDatafor a descriptor) is the substrate for that, and ENS/chain would beAddressLookupimpls. Separate crate, separate issue.