Repository navigation
feat(mcp): cap a run from the MCP surface -- plan and start take max_calls / max_usd - #1186
Conversation
`plan` and `start` take `max_calls` / `max_usd`, so a host can bound a run's model calls per call instead of only through the process-wide AGENTDESCENT_MAX_CALLS. Both validate through `build_call_limit`, the same rule the CLI and the run store use, so a `max_usd` with no price refuses with the CLI's own sentence and a bad number is refused before a run directory exists. The operator's ceiling is applied with the same `min` as `runstore.launch` / `execute`; the reply carries `call_limit` with the enforced number and, only when the ceiling bound it, `source` plus `max_calls_requested` -- the distinction AGENTDESCENT_MAX_CALLS_REQUESTED already carries across the detach boundary. `start` passes the caller's raw numbers to `launch` (which clamps and records the label), never a pre-clamped value, because a clamped requested == ceiling would claim no ceiling bound the run. Fixes #1184.
|
Tip Local at the same tip: The body lists the full |
|
Advisory read at T3 by the size cap only: 4 files Tip SHA: 4015bb8
Findings — one correction and one non-blocking observation, no code defect:
Behaviours I probed myself at the tip:
No UI surface: tool arguments and a JSON block plus a docs row; nothing rendered changes. |
|
Advisory read answered at
Extra probe of the positive dollar path, which the new file does not cover: Awaiting the counting PASS at this tip. |
|
Review (Birfy) Tip SHA: 4015bb8
|
Fixes #1184.
Claimed under AGENTS.md "Claim, PR, release"; agent-ready is triage-side and applied to 0/18 open issues (repo-wide), so it was not waited on.
What changed
The run-level call cap (#1166) and the operator ceiling (#1082) were both unreachable from the MCP surface:
planandstarttookbudget_usd/usd_per_calland nothing else, so a host that wanted "at most N model calls for this run" had no lever except the process-wideAGENTDESCENT_MAX_CALLS, which it cannot lower per call.planandstart(both the tool bodies and themaybe_toolwrappers) now takemax_callsandmax_usd. They validate throughbuild_call_limit, the same rule the CLI and the run store use, so a bad value, or amax_usdwith nousd_per_call, refuses before a run directory exists and with the CLI's own sentence rather than a second wording for this surface.The operator's ceiling is applied with the same
minasrunstore.launch/execute: a caller may lower it, never raise it, and a caller that names no cap still gets it. The reply carriescall_limit:{"max_calls": 3, "max_usd": null}— nothing clamped it.{"max_calls": 2, "max_usd": null, "source": "AGENTDESCENT_MAX_CALLS", "max_calls_requested": 9}— the ceiling bound a request of 9.{"max_calls": 2, "max_usd": null, "source": "AGENTDESCENT_MAX_CALLS"}— the caller named no cap and the ceiling applies anyway.sourceandmax_calls_requestedappear only when the ceiling bound the run, the same distinctionAGENTDESCENT_MAX_CALLS_REQUESTEDalready carries across the detach boundary: without them a host cannot tell "the ceiling lowered your 9 to 2" from "you asked for 2". A request of exactly the ceiling reports not bound, matching_clamped_max_calls.startpasses the caller's own numbers torunstore.launch, not a pre-clamped value:launchapplies the sameminand is what records what was asked for in the child's environment, so handing it an already-clamped number would makerequested == ceilingand erase the label fromstop_detail, while the arithmetic stayed right. The enforced clamp still crosses the process boundary as--max-calls 2, which the test asserts on the child's argv. No argument means exactly today's behaviour, and plainplanhas nocall_limitkey.max_concurrency,doctor, and any USD default for un-priced runs are untouched, as the issue scopes them out: a process-wide dollar ceiling would turn every un-priced run into aSpecErrorthroughrequire_budget_pair.Red test on
f5017f93tests/test_mcp_call_limit.pyalone at cleanf5017f93→ 8 failed, 2 passed. The eight are the missing entry point, not a behaviour disagreement:test_plan_reports_the_cap_it_will_enforce→TypeError: Tools.plan() got an unexpected keyword argument 'max_calls'test_start_caps_the_run_and_status_shows_it,test_max_usd_alone_is_refused_with_the_cli_sentence,test_a_caller_cap_above_the_operator_ceiling_is_clamped_and_labelled,test_a_cap_equal_to_the_ceiling_is_not_reported_as_bound,test_start_hands_the_clamped_cap_to_the_detached_child,test_nested_start_is_unchanged→ the sameTypeErroronstarttest_a_run_that_names_no_cap_still_gets_the_ceiling→KeyError: 'call_limit': the base payload has no such key, because only the new path reports the ceiling it will enforceThe two that pass are the two "no cap, behaviour unchanged" rows (
test_plan_without_a_cap_is_unchanged,test_start_without_a_cap_is_unchanged), which assert today's behaviour and must hold before and after.Provenance: re-measured at the base with only this branch's
tests/test_mcp_call_limit.pycopied into a clean worktree off5017f93—PYTHONPATH=$PWD pytest -o addopts='' tests/test_mcp_call_limit.py -q→8 failed, 2 passed in 2.46s; all eight fail on theTypeErrorfor the newmax_calls/max_usdkwargs, and the two passers are the rows named above. The behaviour rows are asserted green at this tip: the run stops at the cap (stop_reason="budget", the ceiling named instop_detail), the child's argv carries--max-calls 2when the caller asked 9 under a ceiling of 2, a refusedmax_usdleaves no run behind, and nestedstartstill returns its unchanged stub.Clamp teeth, measured: with the
minbypassed in the helper and nothing else changed, the file went 2 failed, 8 passed (test_a_caller_cap_above_the_operator_ceiling_is_clamped_and_labelled,test_a_run_that_names_no_cap_still_gets_the_ceiling). The probe was reverted, not shipped; the file greps clean and is 10 passed.Tip SHA: 4015bb8
Ran:
pytest -o addopts='' -q(the full offline suite) at4015bb8f→ 21 failed, 10 errors, 7924 passed, 51 skipped in 2169.68 s. The 31 red ids are the identical set red on14ba9529, a commit that is five commits older than this PR's basef5017f93— the five being38f9d90f(release v0.11.0),920dc8c3(fix(probe): print one refused-probe contract on the CLI and MCP plan #1178),98655111(fix(cli):snapshot eval --jsonreports"ok": falseon a partial eval (#1179) #1180),a1b41e76(feat(runstore): operator call ceilingAGENTDESCENT_MAX_CALLS(Partial #1082) #1181) andf5017f93(docs(evidence): require every card to name its reproducing command (#951) #1177), i.e. four code merges sit between the baseline and the base. That is the stronger direction: the same ids are red before those merges as well, so neither this branch nor the four merges in between accounts for them; the red ids are docker engine, exec-bit and wheel-smoke environment rows, and the comparison is a same-idcomm(empty both ways), not a count. No red id touchesmcp,plan, the run store caps or this new file. The one count difference from the earlierc6000830measurement istest_dsh_superagent_run.py::[tmux], which is load-flaky (it passes in isolation at both commits) and passed in this run.pytest tests/test_mcp_call_limit.py→ 10 passedtest_mcp_call_limit,test_mcp,test_mcp_meta,test_mcp_plan_probe,test_mcp_http_transport,test_operator_run_cap,test_budget_guard) → 116 passedpython -m tools.changelog_fragments check→5 changelog fragment(s) OKpython -m tools.gen_api_docs --check→docs/api.md is up to datepython -m tools.gen_algorithms_docs --check→docs/algorithms.md is up to datemkdocs build --strict→ rc 0, 12.4 s, with thesocialog-image plugin stripped in a temporary untrackedmkdocs-local.yml(it blocks on a font fetch in this sandbox; CI runs the full config)Not run:
ARK_API_KEYis not in this box's environment; every test row is offline and the cap is arithmetic, not a model calldoctor,max_concurrency→ explicitly out of scope in the issueNo UI surface: this adds two tool arguments, one JSON block and a docs row; there is no UI to screenshot.