Skip to content

Name the node and the reason when a workflow start's run Bots are not created (#3282) - #3284

Open
Birfy wants to merge 2 commits into
mainfrom
dev/3282-the-issue-fast-line-wf-1cdf6ec5-cannot-c
Open

Birfy wants to merge 2 commits into
mainfrom
dev/3282-the-issue-fast-line-wf-1cdf6ec5-cannot-c

Conversation

@Birfy

@Birfy Birfy commented Nov 21, 2026 •

Copy link
Copy Markdown
Collaborator

Fixes #3282

Written-by: Dev
Owner: Dev
Reviewer: Reviewer
Tier: T2 (plugins/** is T2 in .github/agent/risk-rules.json; docs/** and changelog.d/** are T0/T1) — no gate file, so no Policy-sign: line
Base: 202fcc6

What changed

Before: a start whose node Bots the host did not finish creating answered {"ok":false,"error":"the host did not create the run Bots within 60s"} — no node, no reason — and the Bots API delivered that inside a success envelope, {"ok":true,"value":{"ok":false,…}}, so a caller that read the envelope first saw a failed dispatch as a started run.

  1. The refusal names where it stopped. The spawn loop now reports the node it is on and the Bots the host already answered (plugins/workflow/src/service.js, startLive/boundedSpawn), so the deadline sentence reads e.g. the host did not create the run Bots within 60s: 5 of 6 node Bots were created ("Triage" (triage) in 11.0s, …); node "取证" (attest) still had no Bot; a lane of 6 nodes needs about 66s on this host at 11.0s per Bot, past the 60s deadline. A node Bot the host refuses keeps the node and the host's own words, classified model-missing / name-taken / no-slot / no-answer / refused: node "Claim" (claim) could not create its Bot (model-missing): Unknown model opencsitool/Nope-1017. The 60 s deadline itself is unchanged.
  2. The envelope fails when the dispatch failed. workflow/assign is registered through the new service.assignEndpoint(), which throws a refused answer instead of returning it, so api/bot answers {"ok":false,"error":{"code":"bot/refused","message":"…","details":{"workflowId","taskId","nodeId","nodeName","reason"}}} (core/src/host/channel.js ok()/fail()). assign() itself is untouched: { ok: false, error } in-process, never thrown.
  3. Call sites that follow the envelope change (the only ones): the canvas dialog confirmAssign in plugins/workflow/client.js now reads both assign paths through stageCall/settledCall, which turns a refused ok:false answer into the same { ok: false, error } it already drew. In-process callers need no change — the drain, ingest, runNamed, the tools and the plugin's tests keep reading assign(). Nothing else in the repo calls workflow/assign (code search; docs/subsystems/workflow.md now says so).

Why wf-1cdf6ec5 does not start (located, not changed)

Read from the live store ($DSH_HOME/bot/flybot-workflow/workflows.json, read-only; the lane itself is untouched):

  • It is the only 6-node lane (triage, claim, gate, enqueue, escalate, attest); the two controls are 5 and 3 nodes. All six nodes name the same model the working lanes name (opencsitool/DeepSeek-V4.1-Flash-1017), so no model id is missing. Node names are short and the host's runnerName appends a 4-hex suffix and retries on "already exists", so no Bot name is blocking. Its triage node is the only one of the three that carries a seated botId, which only feeds prompt/role/model fallbacks.
  • Node Bots are created serially before the run row exists, and each one costs about 10–12 s on this host: six sessions appear at 06:40:44, :53, 06:41:03, :13, :25, :35 for the first attempt, while the 5-node Triage line took 43 s (06:44:53→06:45:36) and the 3-node implementation line 23 s (06:46:01→06:46:24) and 22 s (07:03:07→07:03:29). 6 × ~11 s ≈ 66 s > the 60 s deadline, so the wait expires with five Bots made, the start is refused, the idle Bots are released and no run is ever written — which is why runs stayed 0 and freeing a tree slot changed nothing.
  • Aggravator, not cause: state.json is 8.1 MB and still carries 329 leaked ephemeral run-Bot records, so every create rewrites the whole team store and per-Bot creation has grown with the roster. Cleaning those records would give the lane back its margin; the durable fix on the owner's side is a shorter line (five nodes or fewer, or Triage/Escalate split out) — that is a live-definition change, left to the owner as the issue asks.

Verification at the tip

Tip SHA: 0f2d6b378be87dcaad57ec07327f3db82c50238b — tree 26f74dce1f16ef809262e8bc69ab563cc9af9cd6, parent = c40d4b829e8fc2f08ba9dc715bebf41e6215561e (the review commit) on base 202fcc6711ac1e8af0877a7ab6865f05088487ae.

Ran:

  • Red first: the new file against the base service.js (implementation reverted, tests unchanged) — 3/3 fail: the node that failed is named: Unknown model opencsitool/Nope-1017 (bare host text, no node), the node that never answered is named: the host did not create the run Bots within 60s (bare deadline), okSvc.assignEndpoint is not a function (no seam to refuse through).
  • Green after the change: node --test plugins/workflow/test/spawn-failure-message.test.js 3/3, including the control that assign() keeps answering { ok: false } and never throws.
  • node --test "plugins/workflow/test/*.test.js" (this checkout, no npm ci): # tests 540 / # pass 539 / # fail 1 — the one failure is host.test.js, which imports the built ds-bot package this box has no copy of (Cannot find module 'ds-bot'); every other file is green, late-spawn-release.test.js and the new file included, client.test.js 180/180.
  • node --check on service.js, index.js, client.js and the new test; node .github/scripts/changelog.mjs check — 2 fragment(s) in changelog.d/ are well formed.
  • The sentence the real failure would now print, produced by running the six-node graph against a clock that advances 11.0 s per Bot: the host did not create the run Bots within 60s: 3 of 6 node Bots were created ("Triage" (triage) in 11.0s, "Claim" (claim) in 11.0s, "Gate" (gate) in 11.0s); node "Enqueue" (enqueue) still had no Bot; a lane of 6 nodes needs about 66s on this host at 11.0s per Bot, past the 60s deadline.

Not run (and why):

  • TZ=UTC npm run verify and cd core && npm ci && npm run harness: this box has no git and no full checkout (the tree was fetched file by file over the REST API), so core/ds-bot cannot be built here; the same is why host.test.js fails to load. No core/** file is touched by this PR.
  • node scripts/lint-plugins.mjs: no oxlint found on this box (npm run harness first). npm run arch-docs: the script is not part of the partial copy — docs/subsystems/workflow.md is updated by hand for the changed owned file.
  • node scripts/build-client.mjs --check: no core/src/client/** change.
  • UI screenshots: the canvas refuses through the same { ok: false, error } it already drew; client.test.js 180/180 covers the dialog.

Checkpoint

{
  "pr": 3284,
  "claim": "Dev",
  "issue": 3282,
  "summary": "A workflow start the host could not finish now names the node it waited on, how many node Bots were made, how long each took, and the seconds the lane needs past the 60s deadline; a refused node Bot carries the node and the host's reason. workflow/assign throws that refusal through assignEndpoint, so api/bot answers { ok: false, error }; assign() keeps its in-process answer and the canvas reads it through settledCall. wf-1cdf6ec5 is the only six-node lane: about 11s per Bot here runs past the 60s deadline before a run row exists; the live definition is untouched.",
  "files": ["plugins/workflow/src/service.js", "plugins/workflow/index.js", "plugins/workflow/client.js", "plugins/workflow/test/spawn-failure-message.test.js", "docs/subsystems/workflow.md", "changelog.d/3282.fixed.md"],
  "checks": { "ran": "red first at base 3/3 fail; after the change the new file 3/3 and workflow 539/540 (host.test.js needs the built ds-bot package); client.test.js 180/180; node --check; changelog check; six-node timing simulation", "result": "green", "at": "0f2d6b378be87dcaad57ec07327f3db82c50238b" },
  "next": "none — ready for review"
}

… created

A start whose node Bots the host does not create in time answered only
"the host did not create the run Bots within 60s", so a lane that needs more
time than the deadline looked like a silent hang. The refusal now names the
node the host never answered, how many node Bots it did create and how long
each took, and the seconds the lane needs when that is past the deadline; a
node Bot the host refuses carries the node and the host's own reason.

The endpoint seam refuses by throwing, so api/bot answers the refusal as
{ ok: false, error } instead of putting { ok: false } inside the success
envelope; assign() keeps its own answer for in-process callers and the canvas
reads the thrown refusal through settledCall.

Fixes #3282
Written-by: Dev
@Birfy

Birfy commented Nov 21, 2026

Copy link
Copy Markdown
Collaborator Author

Written-by: Dev

Note on the checkpoint: the enqueue-guard reading at this head (1698 characters, summary 821) was taken from the body as it was opened. The block is now 1203 characters with a 569-character summary — inside both limits, at still naming this tip — and no commit was added, so the rewrite is a body edit only. This comment is what re-judges it: the guard runs on issue_comment as well as on a push.

The remaining reason in enqueue-guard (no Verifier report names the tip c40d4b829e8f) is the T2 gate itself, for the PR 快线 to answer at this tip.

A start waits for its node Bots, and the host refuses it at its own 60 s
deadline, so the canvas must not cut the call at the shared 10 s: both assign
call sites pass ASSIGN_TIMEOUT_MS (70 s) and still show a refused answer as
the sentence the dialog already drew.

Fixes #3282
Written-by: Dev
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agent-pr Pull request opened by an agent

Projects

None yet

Development

Successfully merging this pull request may close these issues.

The issue fast line (wf-1cdf6ec5) cannot create its run Bots: assign fails three times with a bare 60 s timeout

1 participant