Skip to content

mcp/cli: a refused --probe reports ok:true+ready:false on the CLI but ok:false and no ready over MCP plan #1175

Closed

Description

@Birfy

Problem

The same refused probe gets two different JSON contracts depending on the surface a host uses:

surface ok ready exit
CLI agentdescent plan spec.json --probe --json true false 1
CLI agentdescent algorithms run ... --probe --json true false 1
MCP plan(spec, probe=true) (#1170) false (absent) —

Hosts are told (generated SKILL.md, "run the plan with the probe on: agentdescent plan <spec> --probe, or the MCP plan(spec, probe=true) … If it refuses …") that the two are interchangeable, but a host that drives the CLI (Grok Bot scaffold, any host without MCP) must read ready, while an MCP host must read ok, and neither key means the same thing on the other surface. On the CLI ok:false means "spec/plan failed"; on MCP it now also means "plan fine, probe refused". A host porting its check from one surface to the other silently mis-reads a dead subscription as ready (CLI ok:true) or a probe refusal as an invalid spec (MCP ok:false).

Reproduction (main 6b034fd, real Volcengine Ark Coding endpoint whose CodingPlan has expired → HTTP 400 InvalidSubscription)

agentdescent algorithms run skillopt prompt.md cases.jsonl --agent openai_compatible \
  --model deepseek-v4-flash --base-url https://ark.cn-beijing.volces.com/api/coding/v3 \
  --key-env ARK_API_KEY --rounds 1 --workers 1 --probe --json
# -> {"ok": true, "ready": false, "next": "renew the subscription at the provider, ..."}  exit 1
agentdescent plan agentdescent.skillopt.spec.json --probe --json
# -> {"ok": true, "ready": false, ...}  exit 1
# MCP stdio client, tool `plan`, {"spec": <same spec dict>, "probe": true}
# -> {"ok": false, "probe": {"ok": false, "rows": [{"state": "failed", "detail": "SubscriptionInactive: ..."}]}, "next": "renew the subscription ..."}   # no "ready"

next and probe.rows agree on both surfaces; only the top-level verdict keys diverge.

Expected

One documented contract for "plan ran with the probe" on every surface, so a host reads the same key on CLI and MCP.

Suggested fix (one PR, additive)

  • MCP plan (_with_probe in agentdescent/mcp.py): whenever probe=true, also set ready exactly as the CLI does (bool(probe.ok)); keep the current ok:false on refusal for backward compatibility with feat(mcp): let the plan tool run the preflight probe (#1094) #1170, or document why it differs.
  • Document the contract once (e.g. docs/mcp-path-a.md and the probe paragraph of integrations/SKILL.md and its generated copies): "after a probe, read ready; ready:false → give the user next, do not start/evolve".
  • Optional: CLI --probe keeps ok:true (the plan itself is valid); say so in the same doc paragraph.

Acceptance

  • With a 400 InvalidSubscription stub, MCP plan(spec, probe=true) returns ready: false and CLI plan --probe --json returns ready: false; with a 200 stub both return ready: true (tests on both surfaces, e.g. extend tests/test_mcp_plan_probe.py).
  • MCP plan without probe does not grow a ready key (or the doc states what it means there).
  • The SKILL.md probe paragraph names ready for both surfaces; installs for all 5 hosts carry it (existing host-sync test stays green).
  • changelog.d/ fragment added.

Activity

  1. added
    area:pluginsHost plugins and installers: Claude Code, Codex, DSH, OpenCode, MCP server
    area:cliThe agentdescent command line (cli.py, doctor, plan/evolve/status/apply)
    on Nov 20, 2026
  2. added
    agent-workingAn agent has claimed this issue (the claim comment names the branch)
    bugSomething isn't working
    on Nov 20, 2026
  3. HugoXK commented on Nov 20, 2026

    @HugoXK
    Collaborator

    Claim: agent/1175-probe-contract — one PR against main, maintainer_can_modify: true. Contract: a refused probe prints ok: false + ready: false + next on the CLI and over MCP.

  4. HugoXK commented on Nov 20, 2026

    @HugoXK
    Collaborator

    PR: #1178 (draft) — issue + PR paired; final tip follows.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

agent-workingAn agent has claimed this issue (the claim comment names the branch)area:cliThe agentdescent command line (cli.py, doctor, plan/evolve/status/apply)area:pluginsHost plugins and installers: Claude Code, Codex, DSH, OpenCode, MCP serverbugSomething isn't working

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions