Problem
abuse.tiers suspends a key once its daily count of admission rejections crosses a tier. The counter (note_abuse, crates/handler/src/lib.rs) increments on every STOP_LIMIT_MSG 429, and limit_denied (crates/dag/src/nodes.rs) raises that one code for five different limits:
| Node |
Scope of the limit |
Whose traffic fills it |
tenant_rate |
tenant QPS |
every key of the tenant |
rate_limit |
key QPS |
the key |
product_qpm |
product QPM |
every key of the product |
model_qpm |
model QPM |
every key calling the model |
ak_tpm |
key TPM |
the key |
Three of the five are pooled. A key that sends one request a second can collect 429s all day because a sibling key saturates the tenant, product, or model bucket, and those 429s count toward its own suspension. The sanction lands on a key that did nothing the limit was written to stop.
The order makes it worse: tenant_rate runs ahead of rate_limit, so when a flooding key saturates the tenant bucket, the flooder's rejections and the bystanders' rejections are indistinguishable — both are tenant-QPS denials.
Proposal
- Count only the limits the key owns: key QPS and key TPM. Pooled-scope denials (tenant QPS, product QPM, model QPM) get their own internal error code, still class
Throttling, still 429 with the same client-visible shape; note_abuse keys on the key-scoped code only.
- Check key QPS ahead of tenant QPS at the admission sites (the
model_access DAG, and the realtime, MCP and video-poll gates that repeat the sequence). A flooder then trips its own limit first and is counted; a bystander that stays under its own QPS reaches the pooled check and is not.
With both, a flooder cannot hide behind a pooled limit (its own QPS limit fires first), and a bystander cannot be suspended for a sibling's traffic.
Notes
- A key with no
qps of its own and no tokens_per_minute is never counted after this change. That is the intended reading: abuse tiers sanction a key for exceeding its own limits, so a key without limits has nothing to exceed. Document it in docs/configuration.md next to abuse:.
- Reordering changes which bucket a rejected request consumes: today a request denied by tenant QPS never touches the key bucket; after the change a request denied by tenant QPS has already taken a key-QPS slot. State it in
docs/governance.md.
- Hot path: zero added work on an admitted request (same checks, different order); the 429 path is unchanged (one governance call).
Acceptance
- Unit: a pooled-scope denial leaves
abuse:{ak} untouched; a key-scoped denial increments it.
- E2E: two keys in one tenant with a tenant QPS cap and
abuse.tiers set; key A floods, key B sends under its own limit. A is suspended, B is not, and B's 429s during the flood carry the pooled code internally.
- Docs:
configuration.md, governance.md, security.md state which limits count.
Problem
abuse.tierssuspends a key once its daily count of admission rejections crosses a tier. The counter (note_abuse,crates/handler/src/lib.rs) increments on everySTOP_LIMIT_MSG429, andlimit_denied(crates/dag/src/nodes.rs) raises that one code for five different limits:tenant_raterate_limitproduct_qpmmodel_qpmak_tpmThree of the five are pooled. A key that sends one request a second can collect 429s all day because a sibling key saturates the tenant, product, or model bucket, and those 429s count toward its own suspension. The sanction lands on a key that did nothing the limit was written to stop.
The order makes it worse:
tenant_rateruns ahead ofrate_limit, so when a flooding key saturates the tenant bucket, the flooder's rejections and the bystanders' rejections are indistinguishable — both are tenant-QPS denials.Proposal
Throttling, still 429 with the same client-visible shape;note_abusekeys on the key-scoped code only.model_accessDAG, and the realtime, MCP and video-poll gates that repeat the sequence). A flooder then trips its own limit first and is counted; a bystander that stays under its own QPS reaches the pooled check and is not.With both, a flooder cannot hide behind a pooled limit (its own QPS limit fires first), and a bystander cannot be suspended for a sibling's traffic.
Notes
qpsof its own and notokens_per_minuteis never counted after this change. That is the intended reading: abuse tiers sanction a key for exceeding its own limits, so a key without limits has nothing to exceed. Document it indocs/configuration.mdnext toabuse:.docs/governance.md.Acceptance
abuse:{ak}untouched; a key-scoped denial increments it.abuse.tiersset; key A floods, key B sends under its own limit. A is suspended, B is not, and B's 429s during the flood carry the pooled code internally.configuration.md,governance.md,security.mdstate which limits count.