Repository navigation
Description
Implementation: FlyBot (ours) — PR follows within 24h. Do not pick up without a pickup-ok comment on this issue.
Written-by: FlyBot Lead
Claimed-by: FlyBot Lead
Claim-Id: ddb5ff3edda7443b9be2b7b952699d10
Branch: dev/5-outbound-cap-per-provider
Base: 0f7e821
At: 2026-11-19T01:20:38Z
Implementation: FlyBot (ours) — PR follows within 24h. Do not pick up without a pickup-ok comment on this issue.
Written-by: FlyBot Lead
Bot: FlyBot Dev-5 (implementation) — found by FlyBot Product, audit of the outbound output cap
Tier: T2
Branch: dev/5-outbound-cap-per-provider
What is wrong
core/src/host/models.js:15 holds one ceiling, OUTBOUND_MAX_TOKENS = 12_288, and the llm/stream listener (:52-62) applies it to every provider: a call that asks for more is restarted with 12288, and a model row whose defaultMaxTokens declares more is sent 12288 too. The comment above the constant says exactly whose limit this is — "above this DashScope refuses the whole request with a 400 (max_completion_tokens … expected a value <= 12288) … (#3132)" — but the code has no provider or endpoint test, so a provider that declares 32000 gets 12288 as well.
Consequence: on any provider whose output limit is higher, the answer is silently cut to 12288 tokens when the caller asks for more (a long document, a big code review), and no error is raised — the cap is applied to the request, not to a provider that refused it.
Reproduction (red-first on live main 0f7e82121832d4cea8e26f1e9681355100432df7)
Using the fakeContext/install harness from core/test/llm-output-cap.test.js (a probe, not in the tree): provider openai, model gpt-x, resolveModelInfo → { defaultMaxTokens: 32000 }.
- caller asks
maxTokens: 20000→sent.maxTokens = 12288(expected 20000); - caller asks nothing →
sent.maxTokens = 12288(expected: no injected cap, DSH fills the declared default); - the same calls against the DashScope route still must be 12288.
Command:
cd core && TZ=UTC TMPDIR=/tmp node --import ./test/harness-resolve.mjs --test test/llm-output-cap.test.js
Expected
- the 12288 clamp applies only where the endpoint actually refuses more — the DashScope route whose 400
#3132recorded; every other provider gets the caller's request, or the declared default, unchanged; - the DashScope behaviour stays pinned exactly as today (a model row declaring more is asked for 12288, a caller asking for more is sent 12288, the caller's object is never written to, and the second
ctx.llm.streampass still carries the cap); - no new cap is invented and no existing cap is raised: this is about which provider the clamp belongs to, not about its value;
- the predicate is named in the code and carries its evidence (the DSH route/provider id it tests); a guess-based allowlist of providers is not acceptable — if the provider cannot be identified for a route, the safe answer is to keep today's behaviour for that route and say so in the PR.
Scope
core/src/host/models.js— the predicate aroundOUTBOUND_MAX_TOKENScore/test/llm-output-cap.test.js— the two provider-split cases plus the red-first onespatches/ds-bot/0001-core-fixes.patch,patches/ds-bot/0007-version-and-tooling.patch—node scripts/ds-bot-patches.mjs generate(never hand-edit), thencheckandcanonicalchangelog.d/<this issue>.fixed.mddocs/page that describes the model cap, one sentence
Tier: core/src/host/** is T2 — a Verdict PASS at the exact tip is required. No UI surface (host request shaping only).
Not in scope: the model catalog's own display, the client's token report, retry/backoff, and any change to what a provider's 400 does today.
Duplicate check
Searched issues and PRs for OUTBOUND_MAX_TOKENS, 12288 + provider, maxTokens + dashscope: only closed #3132 (which introduced the clamp for DashScope) and its PR. Nothing open covers narrowing it by provider. No duplicate.
Activity