Skip to content

Standard: run each unit's agent in a Boat VM of its own #149

Closed

Description

@aitianshi

Priority

extreme. A new backend for runs of the standard: where an agent's commands execute.

Components

  • app/nullius (backend)
  • app/web (routes)
  • templates and static files
  • docs and the site
  • the standard (ports, kits, relay, runs)

Today

A standard run's agent runs either as the user on this machine (local processes, with an explicit yes to running unsandboxed) or in a container, which needs Docker or Podman here. A machine without either has no sandboxed way to run an agent.

Wanted

A third backend, Boat: each unit's agent or capability runs in a boat.dev VM of its own, started from this machine with a key saved in Settings → Sandbox. The VM's only way out is the unit's relay on this machine, so the agent never holds a model key. The VM is deleted whatever ends the unit. The builder's Where step offers it with a VM size, and greys out what the account cannot start.

Acceptance

  • nullius standard run ... --backend boat [--boat-size SIZE] runs each unit's agent or capability in its own Boat VM, and conform takes --backend boat
  • The agent in the VM reaches only its relay; no model key is sent to the VM
  • Every VM a run starts is deleted, on success, failure, timeout and Stop
  • The VM's auto-stop follows the unit's time limit, within the 2 hours a free trial allows
  • Time spent starting the VM and installing the agent is not counted as the agent's time
  • Nothing under the user's own private folders (home, keys, Nullius config) is sent to a VM through a link
  • Settings → Sandbox saves, edits and removes the Boat key; Remove asks first
  • The builder's Where step offers Boat only on this machine, with the account's VM sizes; Review refuses a size the account cannot start
  • A run says when the account can start fewer VMs this hour or today than the run has units
  • Units that need a benchmark's MCP server or their own image end as not run on Boat, with the reason
  • docs/standard.md, docs/builder.md and docs/cli/run.md describe the backend

Dependencies and related

Activity

  1. self-assigned this
    on Nov 16, 2026
  2. added
    enhancementNew feature or request
    P2Planned: limited in scope, has a way around it, or a missing feature
    on Nov 16, 2026
  3. aitianshi commented on Nov 17, 2026

    @aitianshi
    CollaboratorAuthor

    Scope update from the review of #150:

    • On Boat, a check that executes the agent's code (score.runs_code: HumanEval, the workspace kit) runs in this machine's throwaway check container, so such a run needs Docker here and is refused without it.
    • A unit whose setup gives the agent MCP servers (the tools kit, τ³-bench) is not run on Boat: they start from this machine's paths.
    • The Boat key is withheld from every other command Nullius runs, and a VM Boat will not delete is reported (delete_error in agent/boat.json).
  4. added a commit that references this issue on Nov 18, 2026
    2815e65
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

P2Planned: limited in scope, has a way around it, or a missing featureenhancementNew feature or request

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions