Repository navigation
Conversation
API modules declare their limits and how to read a throttled response. The Requester waits in process for a short wait and throws RateLimitError with retryAt for a long one; the queue worker defers the message instead of spending a delivery. The last-attempt signal stays the fallback. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
✅ Deploy Preview for friggframework-org canceled.
|
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 754b69ea39
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
There was a problem hiding this comment.
I reviewed this PR and didn't find any bugs — the change is a new ADR document plus one index-table line, with no runtime code affected. Because it proposes a large, multi-phase architecture change (Requester, queue worker, IAM, module contract, run-state, and pacing) with several open questions still unresolved, a human architectural review would still be worthwhile before this is accepted as policy.
What was reviewed: spot-checked the ADR's file:line citations against the current source (requester.js backoff ladder and 4xx handling, backend-utils.js halt logic, Worker.js/queuer-util.js DelaySeconds support, base-definition-factory.js IAM actions) and they match current behavior; also reviewed the phase-2a per-entity pacing design for cross-tenant impact (noted as a ruled-out candidate issue, not a defect in this docs-only change) and the README index-table addition, which is a simple one-line add.
Extended reasoning...
The diff is documentation-only: a new 349-line ADR (docs/architecture-decisions/049-rate-limit-awareness.md) proposing rate-limit-aware handling across the Requester, queue worker, IAM, module contract, and pacing, plus a one-line index update in README.md. No source, tests, or config changed, so there is no injection/auth/data-exposure surface exercised. Spot-checked several of the ADR's file:line evidence citations (requester.js backoff ladder, backend-utils.js halt logic, Worker.js/queuer-util.js DelaySeconds, base-definition-factory.js IAM policy) against actual source and found them accurate. The decision to defer rather than approve is driven by the size and consequence of the proposed design (multi-phase, touches core/devtools/module-library, 8 open questions) rather than any code defect — this is exactly the kind of large architectural decision a human should weigh in on even when the PR itself is just prose.
API modules (headers and body reason, docs-only limits, a non-Requester client, a module that declares nothing), the Requester throttled branch, a Retry-After parser, RateLimitError, the halt rule, Worker.run deferral, an integration handler and the RATE_LIMITED message. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
There was a problem hiding this comment.
This automated review ran again and found no bugs (docs-only ADR proposal, no source code changed). Since the last review, the PR grew substantially with a new appendix of code sketches (~276 added lines): Requester throttled-branch handling, RateLimitError, the queue-worker defer() fallback chain (send/scheduler/ChangeMessageVisibility), the integration handler, and the RATE_LIMITED message shape. Given the size and the number of framework-wide design decisions being proposed (retry/backoff behavior, SQS visibility/IAM changes, run-state and user messaging), a human architectural review is still worthwhile before this ADR is accepted.
Beyond the reported candidate issue, I confirmed it against the current text: in the defer() sketch (A.9), the ChangeMessageVisibility fallback branch calls changeVisibility(record, ...) on the original record rather than persisting next (which holds the incremented deferrals counter), so a redelivery via that path would not see the incremented count — but this is illustrative pseudocode in the ADR appendix, not shipped code, so it doesn't affect any running system.
Extended reasoning...
Docs-only change: a new ~625-line ADR (049-rate-limit-awareness.md) proposing rate-limit handling across the Requester, SQS queue worker, IAM permissions, and run/user messaging, plus a one-line README index update. No source code or security-sensitive surface is touched. Since the prior review, the author appended a large code-sketch section (276 lines); the size and cross-cutting design scope (not any bug) are why a human look is still warranted, and the previously flagged candidate issue is confirmed present but immaterial since it's only illustrative pseudocode.
Records the answers to open questions 1 to 4 (Retry-After for every module, a 5 minute in-process cap, a new message for a wait up to 15 minutes and the scheduler beyond, 10 deferrals and 24 hours) and brings the text and the sketches in line with the code in #656 and #657. Changes to the text: - the hint wins; the fixed ladder is used only when there is no hint, and it has no budget; - classify runs for a 429 and any other 4xx or 5xx except 401, not for a 2xx, and may name only a reason; - the invocation deadline lives in its own AsyncLocalStorage; - the handler checks delivery.isLastAttempt first, then rethrows, because the visibility tier and a cap use a receive; - the RATE_LIMITED message has no `scheduled` flag and is deduplicated; - the base role gets sqs:ChangeMessageVisibility in every stack. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The Requester throws RateLimitError only when a real hint exists, so `source` is never `backoff`. The error also carries `waitMs`. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
There was a problem hiding this comment.
Nothing blocking. The comments below are optional suggestions. There is no need to push a fix for them before merging.
Beyond the inline findings, I also looked at the RATE_LIMITED notification dedup rule (line 269), which keys the 60s skip on module rather than the declared scope key (entity/credential/app, line 132). Since the delegate flows through a per-integration-instance path (module.js:210 → integration-base.js:857), this dedup does not appear to cross distinct entities/credentials in practice — it reads as an underspecified detail in a proposal doc rather than a confirmed bug, so I'm not raising it as a separate finding.
Extended reasoning...
This run's inline findings (three CONFIRMED issues in the ADR's appendix code sketches) already signal that human review is warranted, so no approval is possible. Beyond those, I independently checked the RATE_LIMITED notification dedup keying concern raised as a candidate this run and concluded it is not a clear bug given the per-instance delegate path, distinct from the already-confirmed appendix issues, so it's worth a brief note rather than silence or repeating prior feedback.
`Process.context.rateLimit` is one value per process, so it holds the latest known wait. A new wait replaces it only when its retryAt is later, a WAITING value never hides an EXHAUSTED one, and the worker clears it only when it is WAITING and its retryAt is past or within 5 s. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
There was a problem hiding this comment.
Nothing blocking. The comments below are optional suggestions. There is no need to push a fix for them before merging.
Beyond the inline findings, I checked the deferral-cap math at lines 218-222 (default maxDeferrals=10, maxDeferredMs=24h): a daily/monthly quota wait resolves via a single scheduler-tier defer at retryAt (line 210-213), not repeated 900s delay-tier cycles, so it isn't silently exhausted after ~3 cycles as the candidate issue suggested — the "no longer DLQ'd" claim holds for that scenario.
Extended reasoning...
Docs-only ADR change (no code); this run's inline findings (RateLimitError.create not overridden, defer() missing required exclusion checks, scheduler idempotency gap) already flag the appendix code sketches for human attention. I additionally verified the previously-flagged deferral-cap/DLQ concern and found the scheduler tier handles long waits in one shot rather than via repeated 900s cycles, so that particular concern doesn't hold.
Still open from earlier reviews (3):
- Unresolved: 3 minor or pre-existing.
…docs HubSpot resets a daily limit at midnight in the account time zone, which the module does not know, so the sketch probes every hour instead of computing a UTC midnight. A burst limit names only its reason and takes its time from Retry-After or the burst window. Salesforce uses one error code for the daily limit and for too many concurrent long-running requests, so the sketch reads the message to tell them apart, and tells the delegate before it throws. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
There was a problem hiding this comment.
An organization admin can add extra usage credits at claude.ai/admin-settings/usage.
Once credits are available, reopen this pull request to trigger a review.
- The in-process budget keeps a reserve for the next request of the request timeout or half the time left, whichever is smaller. With the full request timeout the budget was 0 in every 29 s HTTP Lambda. - The RATE_LIMITED message is written only for a wait longer than the in-process cap, and each stored message type keeps its newest 50 items. - The default deferral caps are 30 deferrals and 26 hours, so hourly probes of a daily limit last one full day. - The visibility tier keeps a margin: SQS counts the 12 hour maximum from the receive, and an extension does not reset it. - classify() must return a reason or a time, synchronously; Retry-After is read on a 429 and on a response that classify() recognises; the static step takes the larger of minRetryAfterMs and the window. - backOff: [] means no in-process waits for a hint either; a hinted wait logs at TRACE and the Requester does not log a throw. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
There was a problem hiding this comment.
If your organization's extra usage balance is empty, an organization admin can add extra usage credits at claude.ai/admin-settings/usage. If its monthly spend limit was reached, an admin can raise it on the same page. If neither applies, contact Anthropic support.
Once extra usage is available, comment @claude review on this pull request, or push a new commit, to trigger a review.
ADR-049 section 4 said the run state is cleared only when its retryAt is past or within 5 s. A local end-to-end run showed the gap: a job that resumed before its retryAt finished, and its process kept `status: 'WAITING'`. The deferred body now carries `_frigg.deferralId`, the run state stores it, and the message that set a wait clears it at once; other messages still clear it only after its retry time (#657). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
seanspeaks
left a comment
seanspeaks
left a comment
There was a problem hiding this comment.
Proposed ADR-049 amendments that follow from the #656 review. Drafted with Claude Code at Sean's request; the direction is Sean's.
Direction: an API module describes its provider. The caller decides how long it may wait in process. A long wait becomes an event, and the worker or orchestrator decides what happens next. Most of the ADR already fits: phase 2's deferral is the right mechanism. These changes move the decisions to that shape.
§1 API-module contract
- Keep
scope,windows,maxConcurrency,minRetryAfterMs,parsers,classifyanduserHints. All of these describe the provider. - Say explicitly that a module never decides waiting, retrying or deferral.
- A
classifyhint for a status other than 429 must not fall back to thebackOffladder. If it has no time (from the parsers,windows.resets,minRetryAfterMsorperMs), the Requester throwsRateLimitErrorimmediately withretryAtunknown, and the worker handles an unknownretryAt(see §3). Today this falls through to the unbudgeted ladder (#656, finding 1).
§2 Requester
- Move
maxInProcessWaitMsout of the module policy into the invocation context, set by the entry point:runWithInvocationDeadline({ deadlineAt, maxInProcessWaitMs }). Defaults per entry type: HTTP ~10 s, queue worker small (0 once phase 2 lands, so workers always defer instead of holding one of the 20 slots), admin/script runners their own. - List every entry point that must set the deadline: HTTP, queue workers, admin script and report routers/executors, db-migration. Several don't today.
- Bound hinted retries:
max(backOff.length, N)ormaxHintedRetries, and say that hinted retries don't use up the ladder'sattemptbudget for 5xx/ECONNRESET. - Propagate the rate-limit fields through the OAuth token-endpoint path (
_transportFailureError).
§3 Queue worker → event, then a default handler
Replace "Worker.run defers a record" with:
Worker.runcatchesisRateLimitedand dispatches aRATE_LIMITEDintegration event with{ retryAt, waitMs, reason, module, scopeKey, message, delivery }.- If the integration defines a handler for it, that handler decides: defer, end the run, notify, or rethrow.
- Otherwise a framework default handler applies the tiers already written here:
DelaySecondsup to 900 s, the scheduler beyond that,ChangeMessageVisibilitywithout a scheduler, plus themaxDeferrals/maxDeferredMscaps. - An unknown
retryAtgoes to a configured probe interval (e.g. 1 h for a daily limit) instead of the visibility timeout. - ADR-047: the same
RATE_LIMITEDevent is what the orchestrator's reducer receives later, so this interim mechanism becomes the orchestrator's input rather than a parallel path. Cross-reference it in ADR-047's event types too.
§5 User-facing message
The RATE_LIMITED message should be written by the worker/default handler from the event, not by a delegate from the module up to IntegrationBase. The module still supplies the text and links as data (userHints). That also settles the open "the Requester doesn't know whether the worker will defer" problem, because the component that decides is the one that writes.
§7 Phasing
Phase 1 alone changes the behaviour for a daily-limited 403: it used to be discarded at once, and now it's lost after about an hour via the DLQ (3 receives about 30 min apart with the generated VisibilityTimeout: 1800 / maxReceiveCount: 3). That's slightly better, because it's recoverable from the DLQ, but it's still lost.
- Either land the event and the default handler (phase 2) in the same release as phase 1,
- or keep the halt exemption out of phase 1 until phase 2.
Decisions to record: who owns the in-process budget (the caller), the RATE_LIMITED event as the single seam for deferral and messaging, and the cap on hinted retries.
|
ADR reconciliation needed after Stack A/B — drafted with Claude Code at Sean's request Severity: cross-ref only. Stack B (#692–#700) adds a new caller of rate-limited modules. It fits the amendments already proposed here (module
|
Frigg 2.0.1 ADR review (COMMENT only — not an approval)Verdict (would approve): nothing blocking. Checklist verified: API modules describe rate limits ( No new test failures vs |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
…wareness # Conflicts: # docs/architecture-decisions/README.md
- The API module describes its provider and never decides waiting, retrying or deferral. A classified limit that is not 429 and has no time throws RateLimitError after one call. - The caller owns the in-process budget through runWithInvocationDeadline. List the entry points that set it and the admin-script and migration ones that do not yet. - Cap hinted retries at 10, off the ladder's attempt. Keep the rate-limit fields on the OAuth token-endpoint error. - The queue worker builds a RATE_LIMITED event. The integration's handler returns defer, drop or retry, or the default handler runs the deferral tiers, with a probe interval for an unknown retryAt. - The default handler writes the RATE_LIMITED message with an atomic upsert. Run state, the halt exemption in phase 2, and the scheduler role for external queues follow the review. - Cross-reference ADR-047, ADR-055, ADR-063 and the sync engine. - Replace the stale core sketches in the appendix with pointers to the code. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
There was a problem hiding this comment.
An organization admin can add extra usage credits at claude.ai/admin-settings/usage.
Once credits are available, someone with write access to this repository can comment @claude review on this pull request to trigger a review.
|
Proposes rate-limit awareness for Frigg: API modules say what their provider's limits are and how to read a throttled response, so Frigg waits exactly as long as needed instead of a fixed backoff and three 30-minute redeliveries.
Why
Decision (summary)
static rateLimit(windows,minRetryAfterMs,maxConcurrency, built-in parsers, aclassify()hook for non-429 signals,userHints).RateLimitErrorwithretryAt. Modules that declare nothing keep today's behaviour.DelaySecondsup to 900 s, the one-time scheduler beyond,ChangeMessageVisibilityas a fallback; capped. feat(core): tell queue handlers their last delivery attempt #653isLastAttemptstays the fallback.context.rateLimit("waiting until X") and aRATE_LIMITEDdelegate with "retry when ready" and provider links.rateLimitedUntilon the entity); no distributed limiter for now (ADR-031).Phased into 5 PRs; the first is core-only and opt-in. Eight open questions, then an appendix of code sketches: four API-module shapes (headers + body reason, docs-only limits, a non-Requester client, a module that declares nothing), the Requester's throttled branch,
RateLimitError, the halt rule,Worker.rundeferral, an integration handler and theRATE_LIMITEDmessage.Related
ADR-031, ADR-042, ADR-047 (#648), ADR-048, #653, #633.
Implementation
RateLimitError: feat(core): honour Retry-After and let API modules declare rate limits #656RATE_LIMITEDmessage: feat(core): tell the integration when a rate limit is too long to wait for #658 (stacked on feat(core): honour Retry-After and let API modules declare rate limits #656)The decisions on open questions 1 to 4 are recorded in the ADR, and the sketches now match the code.
🤖 Generated with Claude Code