Repository navigation
Conversation
Each member works in its own agent sandbox (prover-1..N) and reaches the shared comparator by name; the shared-memory board stays the only channel. The submit tool records the claim per member (keyed by the member's tool state scope), an in-member gate check scores that member's own sandbox, and the final score checks every member's sandbox: CORRECT if any passes. Concurrent checks on one comparator sandbox are now serialized: each check resets the comparator's shared staging directories, so two members checking at once would clobber each other's inputs. Swap the erdos74-swarm8 config for erdos74-swarm4: 4 members, $400 cap, no checkpointing (inspect_swarm does not support resuming a swarm).
… one Each member prover-N now gets its own comparator-prover-N sandbox, the only one it can see, so concurrent checks never share a comparator and the lock goes. The checker picks the comparator for the caller's member (from its tool state scope), and final scoring runs each member's check inside that member's scope so it resolves the same comparator and claim. Single-agent runs keep the single default + comparator pair.
Swarm members' resources tool now shows their own share of the swarm's cost and token caps instead of the sample's limits.
This reverts commit c933fe2.
The boltons resources tool reads sample_limits(), which shows the sample's cost limit rather than the member's share of the swarm's cap. Swarm members get inspect_swarm's swarm_resources() instead, under the name the prompt uses (resources); single agents keep the boltons tool.
agent_type is back to the loop alone (react or deep), and a swarm's members use it too. A positive swarm_size runs a swarm of that many members; None, zero or negative runs a single agent.
swarm_tools() gives no tools and swarm_on_continue() defers to its inner hook outside a swarm, so one builder serves both; only the budget tool differs (resources, or swarm_resources for a member's own share).
Joining every member's name made a path component too long for the filesystem at 32 members, so the task failed to load.
Working time stalls when many concurrent members wait, so it suits a swarm poorly; the swarm also turns a time limit into a soft-stopping cap.
The whole-sample score now states the swarm's size and which members' work passes, then one line per member: verdict, comparator stage, claim, and whether the claim was the default and whether the member submitted. The scorer also stops storing the default "proof" claim: Store.get with a default writes it, so members that never submitted looked as if they had claimed "proof".
One row per member: result, claim, stage, and the last line of the checker's output, which says why a member's work failed.
Adds the member's swarm status (submitted, stopped, share, cancelled, ...) as a column, and the swarm's stop reason, plus what ended it from outside when anything did, to the summary line.
max_file_bytes (262,144) and max_board_bytes (16,777,216) were above the bounds shared_memory() accepts, so building the swarm raised ValueError.
Members no longer get notices injected between turns; they learn of board changes from the status line ending each shared_memory result and from swarm_wait.
0.2.0 removes swarm notices, swarm_on_continue() and member(delivery=...), so drop them (members already used poll delivery). It also removes the shared_memory caps, so restore the intended max_file_bytes and max_board_bytes.
tadamcz
marked this pull request as ready for review
October 28, 2026 15:55
Each member's scored tars go under artifacts/<uuid>/<member>/ with its own attempt counter, so the artifacts show whose work each tar is.
Its UV_EXCLUDE_NEWER predated the release, so the package could not resolve.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds
swarm_sizeto the four tasks. A positive value runs the sample as an inspect_swarm swarm of that many members;None, zero or negative runs a single agent as before.agent_type(reactordeep) still chooses the loop, for a single agent and for every swarm member.How a swarm runs
prover-1…prover-Neach work in their own agent sandbox and check proofs with their own comparator sandbox (comparator-prover-N), the only other sandbox they can reach. The sandbox config writers generate one agent/comparator pair per member.swarm_waitresults; inspect_swarm 0.2.0 injects nothing between turns.cost_limitand split equally (Budget(split="equal", reserve=0.01)): each member stops at its share of 99% of the limit, and the swarm stops at 99% of the time limit. Members getswarm_resourcesas their budget tool, because the boltonsresourcestool shows the sample's limits, not the member's share._proverbuilder:swarm_tools()does nothing outside a swarm.Scoring
submit_proofrecords the claim per member, keyed by the member's tool-state scope. A gate check inside a member scores that member's own sandbox with its own comparator.submit_proofis checked under the default claim"proof", as a single agent is, so an unsubmitted disproof is not credited.Changes that also affect single-agent runs
"proof"claim into the store (Store.get(key, default)stores the default), so a log only shows claims the agent actually made.SandboxComparatorno longer takessandbox_name; it uses the caller's comparator (comparatoroutside a swarm).single-agent/orswarm-<N>/subdirectory.inspect-swarm(0.2.x) becomes a runtime dependency, and the devinspect-aifloor rises to 0.3.283, which inspect_swarm requires.Running a swarm
configs/erdos74-swarm4.yamlis an example. Swarm configs should settime_limitrather thanworking_limit(the config's comment explains why) and should not enable checkpointing: inspect_swarm does not support resuming a swarm.Known limitations
resourcestool could report a member's own budget throughswarm_budget(), which would remove the builder'sswarmargument (TODO inapn/solver.py).defaultservice and nox-default: true, which Inspect's Docker backend requires, so a swarm sample fails at setup under Docker.submission_contentsmetadata key, so it holds whichever member was scored last. The final score writes each member's tree tosubmission_contents_<member>, butscripts/extract_plaintext.pyreads onlysubmission_contents, so it can extract a different member's files than the one whose work passed.