You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Standard: a trial records no agent setup, and its agent time carries the agent's start #180
extreme. The standard's unit loop and the trial it writes (app/nullius/standard/run.py, job.py).
Components
app/nullius (backend)
app/web (routes)
templates and static files
docs and the site
catalogue (benchmarks, agents, capabilities)
the standard (ports, kits, relay, runs)
CI, workflows and skills
Evidence
A standard run of an agent over ACP on main (c4c082a): nullius standard run agent-task-v1 reference --limit 1 --unsandboxed, against a local stand-in endpoint that answers in 2 s. The trial's result.json:
environment_setup 0.049 s
agent_setup (none)
agent_execution 2.108 s
verifier 0.001 s
The agent's process start, initialize and session/new (which also starts the benchmark's MCP servers) run inside agent_execution: Run.unit stamps the agent's start before drive_agent calls acp.Client.start() and new_session(). So:
the trial page reads "Setup —", and its Agent time carries the start;
an agent that never answers its start is written as an agent run of the whole wait, with no setup;
the live feed's trial.end sends one duration, agent_run, from the same stamp, and no agent_setup phase.
Screenshot
Reproduction
nullius standard run agent-task-v1 reference --model <any> --limit 1 --unsandboxed
Open the trial: the bar has no Setup segment and its chip reads "—".
Read the trial's result.json: no agent_setup, and agent_execution starts where environment_setup ends.
Expected
A trial of an agent over ACP records its setup (its start and its session) apart from its run, as Harbor's agent_setup and agent_execution are. A start that fails is setup, not a run. The live board is told the same phases and seconds as the trial's result.
Acceptance
A standard trial of an agent driven over ACP records agent_setup, from the end of its environment's setup to its session opening; agent_execution and the trace's window start there.
A start that ends with no session (the agent never answered, exited, or the run was stopped) is recorded as setup, with no agent_execution.
A command agent, a capability, an Original setup and a control record no setup.
A task's internet door opens before the agent's phases and closes after them: neither is in agent_setup or agent_execution.
The live feed emits an agent_setup phase for such an agent, and its trial.end durations are the trial result's windows.
The field guide's phase rows hold for every runner, not Harbor alone.
Dependencies and related
The trial page's rework (#179, open; pull request #173) draws these phases and words their hovers. This is pull request #174. #173 merges first and words this phase's hover; #174 merges after it.
Activity
fix(standard): time the agent's setup apart from its run (ThinkFlowLa…