Skip to content

fix: a non-JSON vendor error body keeps the vendor's status; cut dead items and wrong comments - #128

Merged
CMGS merged 4 commits into
mainfrom
fix/non-json-vendor-errors
Sep 23, 2026
Merged

CMGS merged 4 commits into
mainfrom
fix/non-json-vendor-errors

Conversation

@CMGS

@CMGS CMGS commented Sep 23, 2026

Copy link
Copy Markdown
Collaborator

Four commits from a whole-repo read of main@38927996: two fixes, then the read's cut-list.

Fixes

a3e4d37c — a non-JSON vendor error body keeps the vendor's status. An upstream that answers an error status with a non-JSON body (a CDN or proxy HTML page on 502/504, a relay's plain-text 413/429) failed serde parsing and became a gateway-internal 500 parse … response. The account took a fault and failed over, the model's fallback_models chain never ran (is_upstream_fault ignores a 500 without an upstream status), a client-caused 413 page cooled the account, and the client saw internal_server_exception instead of the documented 424 model_error_exception with original_status_code (docs/api.md, docs/governance.md "Model fallback"). The six body-parse sites now classify such a body through vendor_error at its real status; a non-JSON body on a success status keeps the gateway parse error. Hot path unchanged: only the error branch differs.

26dcb85e — the realtime refusal on the chat path names the model. Since 39c81a2 the 400 interpolated the protocol string, so it always read "realtime model `realtime`".

Cuts (behavior-preserving, no test changed)

  • ca48cfe2 — BillingRecord derives only Serialize, so its seven #[serde(default)] never took effect; the mock's meta.llama arm is unreachable behind the /model/ arm.
  • 740f67fe — comments the code contradicts or repeats: MOCK_B64 is base64 of "MOCKBYTES", not PNG bytes; Account holds seven strings and two vectors; the content_block_delta arm stated one fact twice.

Net: prod −3 lines, comment lines −3, tests +38.

Evidence

  • Failing-first tests: a_non_json_error_body_keeps_the_vendor_status (handler/tests/fallback.rs) failed on main with GatewayError { code: 1000, http_status: 500, message: "parse openai response", original_status: None }; a_realtime_refusal_names_the_model failed with "realtime model `realtime` …".
  • Per commit on macOS: fmt and clippy clean; tests 687/0/4, 688/0/4, 688/0/4, 688/0/4 (passed/failed/ignored).
  • Linux arm64, rustc 1.98.0, with Postgres 16 and Redis 7: fmt and clippy clean, 688/0/4; the Redis/Postgres suites ran (a gwtest_* database and governance keys were created).
  • Live before/after with real vendor keys. A local fake proxy answered an HTML 502 page and a text 413; the fallback targets were real OpenAI and Anthropic models:
case main this branch
chat, proxy 502 page, fallback gpt-4o-mini 500 internal_server_exception 200 served by gpt-4o-mini, requested name echoed
/v1/messages, proxy 502 page, fallback claude-haiku-4-5 500 200
/v1/responses stream, proxy 502 page, fallback gpt-4.1-mini error frame internal_server_exception native stream from gpt-4.1-mini
3× chat, proxy 413 text, fallback configured 500 ×3, account cooling 424 model_error_exception, original_status_code: 413, no fallback, account ok
ledger 0 rows 3 rows, requested → served
  • Full live matrix on the branch: 209/218. All nine failures are vendor-side: Ernie account overdue (2), SiliconFlow balance exhausted (402, 4) plus its rerank 503 once the 402s cooled that account, Brave key invalid (422), and one qwen-plus implicit-cache miss (an A/B/B/A rerun hit 4/4 on both binaries). The xAI and Ollama groups did not run (no key, no local instance).

An upstream that answers an error status with a non-JSON body (a CDN or
proxy HTML page on 502/504, a relay's plain-text 429 or 413) failed
serde parsing and became a gateway-internal 500 "parse ... response":
the account took a fault and failed over, the model's fallback_models
chain never ran (is_upstream_fault ignores a 500 without an upstream
status), and the client saw InternalServerException instead of the
documented 424/429/503 with original_status_code. The six body-parse
sites now classify such a body through vendor_error at its real status;
a non-JSON body on a success status stays our parse error.
…rotocol

Since the wire-type enum became the protocol catalog (39c81a2) the 400
interpolated the protocol string, so every refusal read "realtime model
`realtime`" whatever model the client sent.
… arm

BillingRecord derives only Serialize, so its seven #[serde(default)]
attributes never took effect. Every Llama call rides a /model/ URL, which
the Bedrock arm answers first, so the meta.llama arm never ran.
MOCK_B64 decodes to "MOCKBYTES", not PNG bytes; Account holds seven
strings and two vectors, not six strings; the content_block_delta arm
stated the native-versus-chat delta handling twice.
@CMGS
CMGS merged commit 9f1f652 into main Sep 23, 2026
2 checks passed
@CMGS
CMGS deleted the fix/non-json-vendor-errors branch September 23, 2026 16:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant