Read a gateway 429 from its status, not only its message (Astryx request 539)

[ryandmonk]

Approved by the owner as option C, before any further paid evaluation. It is one narrowly scoped correction to how the Worker classifies a gateway HTTP 429. No provider call was made to build or test it.

What changes and what does not:

• Changed: the Worker's classification of a rejected provider call, its tests, one harness test file, and a paragraph in docs/deployment.md.
• Unchanged:
  • corpus v2, scoring, thresholds and prompts;
  • request bytes, call budgets and block behavior;
  • the harness's code identity, fe2b8e84…cc86d, which covers neither the Worker nor tests;
  • every piece of Astryx evidence, and request #539's conservative ledger hold.

Why

In the Astryx stage-3 run, request #539 was refused by the AI Gateway with HTTP 429, no usage and no cost.

• The Worker classified a busy refusal only from the error message: /\b429\b|too many requests|rate.?limit|capacity/i.
• That refusal's message named none of those words, so the Worker answered 502 provider-unavailable.
• The paced run then stopped on a call that had reached no model, and held $1.12 at worst case.
• The run's own files never recorded the reply; this is inferred from the code paths and the stop's text.

What the binding provides. The AI binding's error carries only a name and a message. The binding does record the HTTP status of every reply it receives, before it throws (workerd src/cloudflare/internal/ai-api.ts):

this.lastRequestHttpStatusCode = res.status;
...
if (!res.ok || !res.body) {
  throw await this._parseError(res); // InferenceUpstreamError(message, name): no status
}

• The field. It is in workerd's source at every release tag checked, from v1.20250601.0 to main. It is not in @cloudflare/workers-types.
• The message. _parseError builds it from the body's internalCode or errors[0]. Failing both, the message is the reply's whole body.
• This PR reads both, with no new network request, gateway-log query, credential or dependency.

What is now classified as busy (503, released by the bench at $0 and backed off)

All of these must hold:

1. The request's only call is the one refused. It reports no usage and names no model, however the 429 is known. A request that made any other call first, such as an unusable output, is never busy.
2. Nothing shows a model result. This is checked at any depth the rejection reaches: its own fields, data, body, error, errors[], response, cause and so on. Every own key counts, enumerable or not, symbols included.
   • None of usage, model, inputTokens, outputTokens, input_tokens, output_tokens, prompt_tokens or completion_tokens is set.
   • No string, the message included, quotes a usage or token key such as "usage":, in any case, its quotes escaped or not (a body quoted inside another). A model's name alone in a message does not count, since a refusal's body may name the model it refused.
   • It counts as showing one when the rejection cannot be read in full:
     • a throwing getter;
     • more than 6 levels, more than 256 objects, or more than 4,096 values;
     • anything that is not a plain object, array or error, such as a Response and its body, a stream, a Map, a buffer or a function.
3. A known status decides alone. Every known status must be a numeric 429. The statuses read are:
   • env.AI.lastRequestHttpStatusCode, taken as this call's only when:
     • the Worker cleared it to null just before the call;
     • no other model call was in flight when the call began, or began while it ran. A call counts as in flight until the binding settles it, even after the Worker's own timeout. workerd never settles the I/O of a request it canceled, so a call stops counting 5 minutes after it began, by when its request can no longer be running;
     • the binding itself rejected the call, never the Worker's 45 s timeout, and never by throwing before it returned a promise.
   • A status on the rejection, or on its response or cause, recursively up to 6 levels. null and absent count as none; anything else is a known status.
4. Only when no status is known does the message decide, exactly as before. That is the compatibility fallback. It covers:
   • a binding that recorded nothing, such as a call that received no reply;
   • a status the Worker could not attribute to this call;
   • an error that carries none.

The busy reply is unchanged: 503 {error: "busy", message, usage: null, calls: [{inputTokens: null, outputTokens: null, outcome: "provider-error", durationMs}]}. The harness already releases that reply at $0 and retries it after its backoff (retryableRefusal), and the app already shows it as busy.

What fails closed (502 provider, held by the bench at worst case)

• Any known status that is not a numeric 429, or statuses that disagree, whatever the message says:
  • the binding records 500, 503 or 200, even when the message names a busy word;
  • the binding records 429 but the error says 500, or the reverse;
  • a 429 whose cause, 6 levels down, says 500;
  • a status given as a string, such as "429", or one that cannot be read.
• A 429 alongside any sign of a model result, including a usage key quoted in the message, escaped or capitalized. Before, a busy-worded one was released; no current binding error has such fields.
• A rejection that cannot be read in full, including one holding a Response, a stream, a Map, a buffer, a function, or more than 4,096 values.
• A 429 on a request that already made a billed call, such as an unusable first output, whatever its message and whether or not a status is known.
  • Before this PR, a busy-worded one with no status known was answered 503 busy, still reporting the first call's usage.
  • The bench held that reply too, so its outcome does not change. A production user now sees "did not return a usable proposal" in place of "busy".
• The Worker's own timeout, and a status left from an earlier reply, or one another overlapping call may have written.
• Unchanged: 500, 502 and 503 without a busy word, transport failures, and unknown rejections (a string, undefined, a null-prototype object).

Limits

• The gateway must answer 429 only before it forwards a request. That is the basis for releasing a 429 at $0. The second review read Cloudflare's documentation as answering gateway rate limits, spend limits and Workers AI capacity with 429s that forward nothing.
  • Retry Requests must stay off on ds-ai-gateway: with it on, a final 429 could follow an attempt that reached the model. The message-based path before this PR had the same exposure.
  • Guardrails and DLP, if ever enabled, must not answer with a 429. By the same reading, a guardrails block (code 2017) comes after the model ran, with its HTTP status undocumented, and DLP answers 400.
  • docs/deployment.md now states this precondition. Neither setting was read for this PR; that needs the owner's dashboard.
• Statuses other than 429 now decide. If the binding ever recorded a gateway busy refusal with a status other than 429, a paced run would stop on it instead of retrying it. That is the fail-closed outcome the owner required. The gateway's rate limit answers 429, so this is not expected; the first window after deployment would show it.
• The field is not documented. If a runtime stops recording it, the status reads as unknown and classification falls back to the message, as before.
• When the status cannot be attributed, the fallback applies. After the Worker's timeout or a client disconnect, a call the binding never settles keeps its isolate from trusting the status for 5 minutes, and a refusal that overlapped another model call is never trusted. Both are classified by their message, as before.
• A persistent 429, such as a spend limit, is retried until the backoff runs out, as busy-worded 429s already were. Each such cell is then recorded as a free provider failure, eligible for a re-run and left out of the pass rate, and the run moves on. Before, one whose message named no busy word stopped the run.
• A status on a cause can establish busy, not only veto it, as the owner asked for nested statuses. A wrapper that retried and kept an earlier attempt's 429 as the cause could mark a later, billed attempt busy. env.AI.run has no such wrapper and gives no cause.
• Whether #539's reply would now read as 429 depends on the binding's own reply carrying the gateway's 429. The owner's gateway log shows 429; the binding's reply status was not recorded, and checking it would take a provider call. Its message was never recorded either: had its body quoted a usage key (a spend-limit body reporting budget usage, say), it would still fail closed, safely.

Tests

Fail-first, round 1. The tests were committed alone in 9c56db5 and run against the unchanged Worker.

• Worker: 6 of the 13 new tests failed and 7 passed. A 14th, added with the fix, also failed on the old Worker.
• Harness: 2 of 3 failed. The #539 shape stayed a 502 held at worst case, and its backoff exhaustion did not end as the message-classified busy refusal does.

Fail-first, round 2: the fixes in 87f70e6, run against aa66527.

• Worker: 4 of the 6 new tests failed. The other 2 pin behavior already right.
• Harness: a 500 whose message names a busy word was released at aa66527, and is now held.

Fail-first, round 3: the review fixes, tests in ac21982, run against 87f70e6.

• Worker: 6 tests fail, each answered 503 busy where 502 is due:
  • a busy-worded 429 after a billed first output, with no status known, and the same hidden by an overlap;
  • usage quoted in the message, the binding's real shape;
  • usage under a non-enumerable or symbol key, in an AggregateError, or in a string;
  • a rejection holding a Response, a stream, a Map or a buffer;
  • a contradicting status 6 levels down.
• Worker: the other new tests pin behavior already right:
  • the depth and size cap;
  • a recorded status that cannot be read;
  • busy words in prose;
  • a refusal's body naming its model;
  • the 5-minute let-go, to the millisecond.
• Harness: a 429 whose message carries usage was released at $0 at 87f70e6; it is now held. A two-call 429 is held both before and after.
• Two expectations change:
  • a test from before this PR, "a 429 after an unusable first output", now expects 502, still with the first call's usage;
  • this PR's round-1 case of a status on a cause's response now uses plain data, since a real Response now fails closed.

Fail-first, round 4: the review fixes, tests in fdba7f8, run against ebc222f.

• Worker: 3 tests fail, each answered 503 busy where 502 is due:
  • a function carrying usage (Copilot's and the review's finding);
  • more values than the search reads;
  • a usage key escaped inside a JSON string, or capitalized, in the message.
• Worker: the other new tests pin behavior already right, each closing a mutant the review found surviving:
  • each usage, token or model key alone, as a field and quoted, and an empty usage object;
  • the depth cap to the level, with plain data at the deepest level still read;
  • an array, a null-prototype object, null result keys, a quoted word without a colon and a longer key such as daily_usage, all still busy;
  • a binding that ignores the clear;
  • a bare 429 in the message is busy, and a number containing 429 is not.

Round 5, tests and docs only, in a862f5c; the Worker's code is unchanged since fffe5d8.

• The values bound, pinned on plain objects:
  • one object with 5,000 keys, and two with 3,000 each, fail closed;
  • 3,500 values stay busy.
• A lone spaced key ({"usage" : 5}) fails closed.
• Against fffe5d8 with each mutant applied, these kill the 6 the review found surviving:
  • no bound;
  • counting objects instead of keys;
  • an overflow that does not fail closed;
  • a per-object count;
  • a bound of 2,048;
  • a regex without \s*.
• The docs now say only the bench retries a busy refusal; the browser shows it, for the visitor to try again.

Final, at this head (a862f5c).

• Worker: all 65 tests pass. That includes the pinned sha256 of env.AI.run's arguments (request bytes unchanged), the Option B per-call-model tests and the capability list.
• Harness: all 41 hosted.test.ts tests pass.
  • The #539 shape is released at $0 ("refused before any provider call"), backed off ("503 busy") and retried, and the retry is settled at its reported usage.
  • When the backoff runs out, it ends exactly as the message-classified busy refusal does.
  • A 429 showing a model result, in fields or in its message, is held ("usage not verified"), as are a 500 and a 429 after a billed call.
• Typecheck and suites: pnpm typecheck passes. pnpm test passes 280 tests in apps/composer and 262 in packages/block-bench, including the request goldens and drift.
• Full suite: pnpm test at fffe5d8 passed in every package but apps/agent (composer-core 279, composer 280, block-bench 262, and the rest).
  • In this container, 3 of apps/agent's src/project.test.ts rediscover tests timed out at vitest's 5 s default while a review ran alongside.
  • Their file passes with a 60 s limit, the slowest taking 8.1 s.
  • apps/agent is identical to main, does not import the Worker, and passes in CI.
• Differential, over the review's 26,370 scenarios (9 call patterns × recorded statuses × 293 rejection shapes):
  • ebc222f → fffe5d8: none moves toward busy; 170 move toward a provider failure, a usage key escaped in the message.
  • main → fffe5d8: the 1,581 that move toward busy are all intended: a numeric 429 the rejection itself carries, or one the binding recorded for a lone call of its own. None is a multi-call request or the Worker's timeout.
  • 87f70e6 → ebc222f: the review found 390 moving toward busy, all a lone numeric 429 nested 4 to 6 levels down response or cause. Statuses are now read as deep as the search; 87f70e6 read 3 levels and left such shapes to the message. The same change makes a 429 contradicted 4 to 6 levels down fail closed. My earlier, narrower matrix (770 shapes) missed these, so its "none moves toward busy" holds only for it.

Review

• Copilot, on aa66527: one finding, fixed in 87f70e6; thread resolved. A re-review of the later head was requested.
• First independent review, of aa66527: nothing blocking. Its should-fix and nits were fixed in 87f70e6.
• Second independent review, of 87f70e6: nothing blocking. It found no realistic path by which a call that reached a model is released at $0. It raised two should-fix findings, both fixed in ebc222f:
  • a busy-worded 429 after a billed call;
  • usage in the message.
    Its nits are fixed too: the traversal, two surviving mutants, the let-go bracket and the docs.
• Copilot, on 87f70e6 and ebc222f: no open threads. Each summary noted one item it had missed earlier:
  • non-enumerable fields, fixed in ebc222f;
  • function-valued fields, fixed in fffe5d8.
• Third independent review, of ebc222f: nothing blocking, and no realistic path to a $0 release of a call that reached a model. Its two should-fix findings are fixed in fffe5d8:
  • functions;
  • no key pinned on its own: 13 surviving mutants.
    Its nits are fixed too: escaped and capitalized keys, a values bound, the depth test, and five unpinned outcomes.
• Fourth independent review, of fffe5d8: nothing blocking, and again no realistic path to a $0 release.
  • Its mutation run killed 90 of 111 mutants; of the 21 left, 12 are equivalent and 2 performance-only.
  • Its differentials show none moving toward busy from ebc222f.
  • Its one should-fix, the values bound pinned only through arrays, and the spaced-key nit are closed in a862f5c.
• Copilot, on fffe5d8: no open threads, and one Low item: the docs said the browser retries a busy refusal. Corrected in a862f5c.
• Fifth independent review, of a862f5c (this head): clear. No finding is blocking or should-fix.
  • The Worker's code is byte-identical to fffe5d8.
  • 96 of 111 mutants are killed. The 15 left are 12 equivalent, 2 performance-only, and 1 at the exact bound, which can only fail closed.
  • Its answer to the owner's question is again no.
  • Three optional docs nits are left for a later docs change:
    • only a paced bench run retries;
    • the browser shows a generic headline with the busy message as the reason;
    • model is not named among the result fields.

Not in this PR

Two known measurement-tooling issues stay separate follow-ups:

• the parity checklist never reads strings inside arrays;
• analyze.mjs misreads runs with more than one invocation.

🤖 Generated with Claude Code

https://claude.ai/code/session_01R6Ax8qvfN3gYUC85oSGnoi

[cloudflare-workers-and-pages]

Deploying with    Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status	Name	Latest Commit	Updated (UTC)
✅ Deployment successful! View logs	dspack-studio-composer	a862f5c	Oct 13 2026, 12:28 PM

[Copilot]

🟡 Changes recommended

Contradictory or nonnumeric structured statuses can still reach the message fallback and be incorrectly released as retryable busy refusals.

1 open finding

• Preserve non-integer binding values for fail-closed status validation · New

What changed in this PR

Updates gateway refusal classification so structured HTTP 429 evidence can produce retryable 503 busy responses.

Changes:

• Adds structured status and model-result evidence handling.
• Adds Worker and ledger integration tests.
• Documents gateway 429 behavior.

File	Description
apps/​composer/​worker/​propose.mjs	Implements structured 429 classification.
apps/​composer/​worker/​propose.test.mjs	Tests classification and concurrency guards.
packages/​block-bench/​live/​hosted.test.ts	Tests retry and ledger behavior.
docs/​deployment.md	Documents refusal handling.

🧠 Review effort: Balanced

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

🔵 Needs a closer look

Non-enumerable rejection fields can bypass model-result detection and incorrectly release a potentially billed call.

0 open findings

1 resolved since last review

• Preserve non-integer binding values for fail-closed status validation

Previously missed (1)

In code that hasn't changed since last review

Traverse non-enumerable properties to avoid misclassifying 429 errors

apps/​composer/​worker/​propose.mjs:442

Object.values only traverses enumerable own properties. A non-enumerable data/body/error field containing usage (or any non-enumerable throwing getter) is skipped, so a recorded 429 can be returned as retryable 503 busy even though the rejection either shows a model result or cannot be read in full. Traverse all own keys, while retaining explicit traversal for inherited wrapper fields, so these cases fail closed as documented.

🧠 Review effort: Balanced

[Copilot]

🔵 Needs a closer look

Function-valued rejection evidence can bypass the fail-closed inspection and incorrectly release a request at zero cost.

0 open findings

Previously missed (1)

In code that hasn't changed since last review

Function-valued data bypasses validation and misclassifies 429 responses

apps/​composer/​worker/​propose.mjs:449

Function-valued fields are skipped before plainData can reject them. Since functions can carry own properties or getters, a recorded 429 containing (for example) data = Object.assign(() => {}, { usage: ... }) is currently returned as 503 busy and released at $0 instead of failing closed. Let functions reach the non-plain-value check.

🧠 Review effort: Balanced

[Copilot]

🔵 Needs a closer look

Cost release depends on unverified gateway settings and an undocumented runtime field, and the deployment documentation also contains an inaccurate browser-retry claim.

0 open findings

Previously missed (1)

In code that hasn't changed since last review

Browser retry behavior is promised but not implemented

docs/​deployment.md:102

The browser does not retry this response: propose() immediately throws a GatewayError for every non-OK response, and the adapter documents that the pipeline stops at the first AdapterOutputError (apps/composer/app/hosted-build.ts:274-300). Only the bench has the retry/backoff loop, so this currently promises browser behavior that is not implemented.

🧠 Review effort: Balanced