Skip to content

feat(mcp): cap a run from the MCP surface -- plan and start take max_calls / max_usd - #1186

Merged
Birfy merged 1 commit into
mainfrom
agent/1184-mcp-call-limit
Nov 20, 2026
Merged

Birfy merged 1 commit into
mainfrom
agent/1184-mcp-call-limit

Conversation

@HugoXK

@HugoXK HugoXK commented Nov 20, 2026 •

Copy link
Copy Markdown
Collaborator

Fixes #1184.

Claimed under AGENTS.md "Claim, PR, release"; agent-ready is triage-side and applied to 0/18 open issues (repo-wide), so it was not waited on.

What changed

The run-level call cap (#1166) and the operator ceiling (#1082) were both unreachable from the MCP surface: plan and start took budget_usd / usd_per_call and nothing else, so a host that wanted "at most N model calls for this run" had no lever except the process-wide AGENTDESCENT_MAX_CALLS, which it cannot lower per call.

plan and start (both the tool bodies and the maybe_tool wrappers) now take max_calls and max_usd. They validate through build_call_limit, the same rule the CLI and the run store use, so a bad value, or a max_usd with no usd_per_call, refuses before a run directory exists and with the CLI's own sentence rather than a second wording for this surface.

The operator's ceiling is applied with the same min as runstore.launch / execute: a caller may lower it, never raise it, and a caller that names no cap still gets it. The reply carries call_limit:

  • {"max_calls": 3, "max_usd": null} — nothing clamped it.
  • {"max_calls": 2, "max_usd": null, "source": "AGENTDESCENT_MAX_CALLS", "max_calls_requested": 9} — the ceiling bound a request of 9.
  • {"max_calls": 2, "max_usd": null, "source": "AGENTDESCENT_MAX_CALLS"} — the caller named no cap and the ceiling applies anyway.

source and max_calls_requested appear only when the ceiling bound the run, the same distinction AGENTDESCENT_MAX_CALLS_REQUESTED already carries across the detach boundary: without them a host cannot tell "the ceiling lowered your 9 to 2" from "you asked for 2". A request of exactly the ceiling reports not bound, matching _clamped_max_calls.

start passes the caller's own numbers to runstore.launch, not a pre-clamped value: launch applies the same min and is what records what was asked for in the child's environment, so handing it an already-clamped number would make requested == ceiling and erase the label from stop_detail, while the arithmetic stayed right. The enforced clamp still crosses the process boundary as --max-calls 2, which the test asserts on the child's argv. No argument means exactly today's behaviour, and plain plan has no call_limit key.

max_concurrency, doctor, and any USD default for un-priced runs are untouched, as the issue scopes them out: a process-wide dollar ceiling would turn every un-priced run into a SpecError through require_budget_pair.

Red test on f5017f93

tests/test_mcp_call_limit.py alone at clean f5017f93 → 8 failed, 2 passed. The eight are the missing entry point, not a behaviour disagreement:

  • test_plan_reports_the_cap_it_will_enforce → TypeError: Tools.plan() got an unexpected keyword argument 'max_calls'
  • test_start_caps_the_run_and_status_shows_it, test_max_usd_alone_is_refused_with_the_cli_sentence, test_a_caller_cap_above_the_operator_ceiling_is_clamped_and_labelled, test_a_cap_equal_to_the_ceiling_is_not_reported_as_bound, test_start_hands_the_clamped_cap_to_the_detached_child, test_nested_start_is_unchanged → the same TypeError on start
  • test_a_run_that_names_no_cap_still_gets_the_ceiling → KeyError: 'call_limit': the base payload has no such key, because only the new path reports the ceiling it will enforce

The two that pass are the two "no cap, behaviour unchanged" rows (test_plan_without_a_cap_is_unchanged, test_start_without_a_cap_is_unchanged), which assert today's behaviour and must hold before and after.

Provenance: re-measured at the base with only this branch's tests/test_mcp_call_limit.py copied into a clean worktree of f5017f93 — PYTHONPATH=$PWD pytest -o addopts='' tests/test_mcp_call_limit.py -q → 8 failed, 2 passed in 2.46s; all eight fail on the TypeError for the new max_calls/max_usd kwargs, and the two passers are the rows named above. The behaviour rows are asserted green at this tip: the run stops at the cap (stop_reason="budget", the ceiling named in stop_detail), the child's argv carries --max-calls 2 when the caller asked 9 under a ceiling of 2, a refused max_usd leaves no run behind, and nested start still returns its unchanged stub.

Clamp teeth, measured: with the min bypassed in the helper and nothing else changed, the file went 2 failed, 8 passed (test_a_caller_cap_above_the_operator_ceiling_is_clamped_and_labelled, test_a_run_that_names_no_cap_still_gets_the_ceiling). The probe was reverted, not shipped; the file greps clean and is 10 passed.

Tip SHA: 4015bb8

Ran:

  • pytest -o addopts='' -q (the full offline suite) at 4015bb8f → 21 failed, 10 errors, 7924 passed, 51 skipped in 2169.68 s. The 31 red ids are the identical set red on 14ba9529, a commit that is five commits older than this PR's base f5017f93 — the five being 38f9d90f (release v0.11.0), 920dc8c3 (fix(probe): print one refused-probe contract on the CLI and MCP plan #1178), 98655111 (fix(cli): snapshot eval --json reports "ok": false on a partial eval (#1179) #1180), a1b41e76 (feat(runstore): operator call ceiling AGENTDESCENT_MAX_CALLS (Partial #1082) #1181) and f5017f93 (docs(evidence): require every card to name its reproducing command (#951) #1177), i.e. four code merges sit between the baseline and the base. That is the stronger direction: the same ids are red before those merges as well, so neither this branch nor the four merges in between accounts for them; the red ids are docker engine, exec-bit and wheel-smoke environment rows, and the comparison is a same-id comm (empty both ways), not a count. No red id touches mcp, plan, the run store caps or this new file. The one count difference from the earlier c6000830 measurement is test_dsh_superagent_run.py::[tmux], which is load-flaky (it passes in isolation at both commits) and passed in this run.
  • pytest tests/test_mcp_call_limit.py → 10 passed
  • focused set (test_mcp_call_limit, test_mcp, test_mcp_meta, test_mcp_plan_probe, test_mcp_http_transport, test_operator_run_cap, test_budget_guard) → 116 passed
  • python -m tools.changelog_fragments check → 5 changelog fragment(s) OK
  • python -m tools.gen_api_docs --check → docs/api.md is up to date
  • python -m tools.gen_algorithms_docs --check → docs/algorithms.md is up to date
  • mkdocs build --strict → rc 0, 12.4 s, with the social og-image plugin stripped in a temporary untracked mkdocs-local.yml (it blocks on a font fetch in this sandbox; CI runs the full config)
    Not run:
  • real-model probe / USD cap against a live provider → ARK_API_KEY is not in this box's environment; every test row is offline and the cap is arithmetic, not a model call
  • wheel smoke → no packaging or release path change
  • doctor, max_concurrency → explicitly out of scope in the issue

No UI surface: this adds two tool arguments, one JSON block and a docs row; there is no UI to screenshot.

@HugoXK HugoXK added area:cli The agentdescent command line (cli.py, doctor, plan/evolve/status/apply) area:plugins Host plugins and installers: Claude Code, Codex, DSH, OpenCode, MCP server enhancement New feature or request labels Nov 20, 2026
`plan` and `start` take `max_calls` / `max_usd`, so a host can bound a run's
model calls per call instead of only through the process-wide
AGENTDESCENT_MAX_CALLS.

Both validate through `build_call_limit`, the same rule the CLI and the run
store use, so a `max_usd` with no price refuses with the CLI's own sentence and
a bad number is refused before a run directory exists. The operator's ceiling
is applied with the same `min` as `runstore.launch` / `execute`; the reply
carries `call_limit` with the enforced number and, only when the ceiling bound
it, `source` plus `max_calls_requested` -- the distinction
AGENTDESCENT_MAX_CALLS_REQUESTED already carries across the detach boundary.
`start` passes the caller's raw numbers to `launch` (which clamps and records
the label), never a pre-clamped value, because a clamped requested == ceiling
would claim no ceiling bound the run.

Fixes #1184.
@github-actions github-actions Bot added risk:T3 Risk tier T3 (core loop, release, process; needs Danyang's click), set by the controller agent-pr Pull request opened by an agent labels Nov 20, 2026
@HugoXK

HugoXK commented Nov 20, 2026

Copy link
Copy Markdown
Collaborator Author

Tip 4015bb8f97454d4853ecaed86804acc70fe944fb: required checks green at that SHA — build, test (3.9) … test (3.13), otel-wire; reviewed tip / risk tier are report-only.

Local at the same tip: tests/test_mcp_call_limit.py 10 passed, the seven-file focused set 116 passed, full pytest -o addopts='' -q 7924 passed / 21 failed / 10 errors / 51 skipped in 2169.68 s — the 31 red ids are the same ids red at 14ba9529, five commits older than the base f5017f93 (docker engine, exec-bit, wheel-smoke environment rows; none in mcp or the run-store caps). changelog_fragments check, both gen_*_docs --check and mkdocs build --strict clean.

The body lists the full Ran: / Not run:. Awaiting the Verifier report at this tip.

@HugoXK
HugoXK requested a review from Birfy November 20, 2026 12:31
@HugoXK

HugoXK commented Nov 20, 2026

Copy link
Copy Markdown
Collaborator Author

Advisory read at 4015bb8f97454d4853ecaed86804acc70fe944fb — from the QA seat, not a counting PASS (.github/agent/review-policy.json counts Birfy only), and I take no merge action.

T3 by the size cap only: 4 files +239/−13 = 252 changed lines, two over the 250-line T2 cap; by path it is T2 (agentdescent/mcp.py). Live in the same read: open, draft: false, mergeable_state: clean, base f5017f93, body Tip SHA: == head, Fixes #1184.

Tip SHA: 4015bb8
Ran:

  • pytest -q -n 8 at the tip → 21 failed, 7912 passed, 63 skipped, 10 errors (288.14s); the same command on base f5017f93 → 21 failed, 7902 passed, 63 skipped, 10 errors. Same 31 red ids id for id: 0 new red, 0 vanished. The +10 passed is exactly the new file's rows.
  • fail-on-main: tests/test_mcp_call_limit.py only, onto f5017f93 → 8 failed, 2 passed (all eight are TypeError: Tools.plan()/start() got an unexpected keyword argument 'max_calls'/'max_usd'; the two that pass are the unchanged-behaviour rows).
  • python -m tools.gen_api_docs --check → up to date; python -m tools.changelog_fragments check → 5 fragments OK; mkdocs build --strict → exit 0, 13.73s (social plugin removed for the offline box).
    Not run:
  • Docker sandboxes, wheel smoke, Ark real-model — the red ids are host state; the cap is arithmetic, no model call.
  • risk_gate.py — no gh on this box; the tier above is computed from .github/agent/risk-rules.json.

Findings — one correction and one non-blocking observation, no code defect:

  1. Correction to the red-first table: the body says 6 failed, 2 passed and lists six rows, but the file has ten rows and at base eight fail — test_a_cap_equal_to_the_ceiling_is_not_reported_as_bound and test_a_run_that_names_no_cap_still_gets_the_ceiling fail there too, both with the same TypeError. It strengthens the fail-on-main, but the number in the body is not what one run of that file at f5017f93 prints.
  2. Non-blocking, same class as fix(spec): a spec's evolve.max_usd is refused by naming --usd-per-call, a flag that cannot help it #1183: the MCP refusal reuses build_call_limit's CLI sentence — max_usd needs usd_per_call: … (pass --usd-per-call on the command line). A host calling the tool cannot pass a CLI flag; the actionable field on this surface is the usd_per_call argument. The field is named, so I would not block on it, but it is the circle fix(evolvespec): name the spec field an unpriced dollar cap needs #1185 just cut for spec authors.

Behaviours I probed myself at the tip:

  • Tools.plan → (self, spec, usd_per_call=None, budget_usd=None, probe=False, max_calls=None, max_usd=None); Tools.start → (self, spec, budget_usd=None, usd_per_call=None, max_calls=None, max_usd=None); both maybe_tool wrappers forward all six.
  • plan(max_calls=3) → call_limit {"max_calls": 3, "max_usd": null}; plan() → no call_limit key.
  • plan(max_usd=0.05) → ok: false, field named; plan(max_calls="x") → ok: false, "must be a non-negative integer".
  • AGENTDESCENT_MAX_CALLS=2: plan(max_calls=9) → {"max_calls": 2, "max_calls_requested": 9, "source": "AGENTDESCENT_MAX_CALLS"}; plan() under the same ceiling → {"max_calls": 2, "source": …} with no requested.

No UI surface: tool arguments and a JSON block plus a docs row; nothing rendered changes.

@HugoXK

HugoXK commented Nov 20, 2026

Copy link
Copy Markdown
Collaborator Author

Advisory read answered at 4015bb8f97454d4853ecaed86804acc70fe944fb; no code change, tip stays as is.

  1. Red-first table — already fixed in the body, and re-measured here: clean f5017f93 with only tests/test_mcp_call_limit.py copied in → 8 failed, 2 passed. The eight are 7× TypeError: Tools.plan()/start() got an unexpected keyword argument 'max_calls'/'max_usd' plus one KeyError: 'call_limit' (the no-cap plan payload); all eight are the missing entry point, none is a behaviour row. The claim note on feat(mcp): cap a run from the MCP surface -- plan and start take max_calls / max_usd #1184 is corrected in place.

  2. Refusal wording — stays the CLI sentence, deliberately: feat(mcp): cap a run from the MCP surface -- plan and start take max_calls / max_usd #1184's acceptance is "returns the CLI's max_usd needs usd_per_call refusal, not a new sentence", and fix(spec): a spec's evolve.max_usd is refused by naming --usd-per-call, a flag that cannot help it #1183 leaves build_call_limit untouched, so an MCP-only wording would be the second contract feat(run): stop a worker run on max_calls / max_usd with a named stop reason #1166 exists to prevent. The actionable field is named on this surface anyway — the refusal's first clause (usd_per_call), both tool descriptions ("max_usd with usd_per_call"), and the docs row. Accepted nuance, not a defect.

Extra probe of the positive dollar path, which the new file does not cover: start(max_usd=0.05, usd_per_call=0.02) → detached run done, stop_reason=budget, stop_detail max_usd: $0.0400 spent, cap $0.0500, reply call_limit {"max_calls": null, "max_usd": 0.05}.

Awaiting the counting PASS at this tip.

@Birfy

Birfy commented Nov 20, 2026

Copy link
Copy Markdown
Collaborator

Review (Birfy)

Tip SHA: 4015bb8
Ran:

  • .github/scripts/risk_gate.py --repo ThinkFlowLab/agentdescent --pr 1186 → T3 (size 252 lines / 4 files > 250 allowed for T2)
  • pytest -o addopts="" -p no:cacheprovider -q over the MCP and cap files (tests/test_mcp*.py, tests/test_mcp_call_limit.py, tests/test_call_budget.py, tests/test_budget_guard.py, tests/test_operator_run_cap.py, tests/test_run_verbs_json_spec_error.py) at this tip → 190 passed, 9 skipped
  • python -m tools.changelog_fragments check → 5 fragments OK
  • python -m tools.gen_api_docs --check → docs/api.md is up to date
  • mkdocs build --strict → exit 0
  • Required CI at this tip → green (build, test (3.9)–test (3.13), otel-wire; deploy skipped; risk tier / reviewed tip report-only)
    Not run:
  • Full offline suite locally — the CI test matrix is the line
  • Docker sandboxes, wheel smoke, a live model call — the cap is arithmetic, and the red ids reported on this tip are host state
    Findings:
  • No actionable defects. plan / start (and the maybe_tool wrappers) take max_calls / max_usd, validate through build_call_limit before create, report call_limit with the enforced number, and hand the caller's raw numbers to launch so the same clamp records what was asked for. A caller cap above AGENTDESCENT_MAX_CALLS is lowered and labelled; a cap equal to the ceiling is not reported as bound.
  • Non-blocking: an unpriced max_usd on this surface still uses build_call_limit's CLI sentence (pass --usd-per-call on the command line). The field is named, so this does not block; it is the same circle fix(evolvespec): name the spec field an unpriced dollar cap needs #1185 cuts for spec authors.
    Verdict: PASS

@Birfy
Birfy merged commit 050235a into main Nov 20, 2026
10 checks passed
@Birfy
Birfy deleted the agent/1184-mcp-call-limit branch November 20, 2026 16:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agent-pr Pull request opened by an agent area:cli The agentdescent command line (cli.py, doctor, plan/evolve/status/apply) area:plugins Host plugins and installers: Claude Code, Codex, DSH, OpenCode, MCP server enhancement New feature or request risk:T3 Risk tier T3 (core loop, release, process; needs Danyang's click), set by the controller

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(mcp): cap a run from the MCP surface -- plan and start take max_calls / max_usd

2 participants