Repository navigation
fix(web): re-dispatch a webhook call when it is retried - #516
polylane[bot] wants to merge 1 commit into
Conversation
Co-authored-by: polylane[bot] <277585245+polylane[bot]@users.noreply.github.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
Important Review skippedBot user detected. To trigger a single review, invoke the
|
Deploying usesend with
|
| Latest commit: |
3c7e57a
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://b72a9a7a.usesend.pages.dev |
| Branch Preview URL: | https://polylane-autofix-3ecm3nov2vy.usesend.pages.dev |
|
Thanks, the bug is real, and I reproduced it against real Redis. I'm replacing this with #519 for one production reason.
#519 leaves new calls on a plain |
…#519) * fix(web): make webhook Retry deliver a call that used up its attempts BullMQ ignores add() when a job with the same id exists, and webhook jobs are kept for 30 days after they fail. So pressing Retry on a call that had used up its attempts reset the row to PENDING but queued nothing. Retry now goes through requeueJob: a kept failed job is moved back to the queue with its attempts reset, in one atomic step; a job that is still waiting, delayed or running is left alone; otherwise a new job is added. New calls keep the plain add(), so the hot path does no extra Redis work. Supersedes #516, which removed the job before every add(), including for every new call, where BullMQ's remove scans the wait list. Co-authored-by: Polylane <polylane[bot]@users.noreply.github.com> * test(web): keep the requeue integration tests from hanging or timing out on slow CI * fix(web): keep a webhook call retryable when Retry can't queue it * fix(web): only undo Retry's own reset when queueing fails --------- Co-authored-by: Polylane <polylane[bot]@users.noreply.github.com>
Retrying a failed webhook delivery returned success but sent nothing. The retry reused a job id that BullMQ still held for the failed attempt, so the new job was dropped and the call stayed pending. The retry now queues a fresh job, so the subscriber receives the event. BullMQ ignores add() when a job id already exists, and failed jobs are kept for 30 days, so the retry was dropped. The resolved BullMQ add script and the retained-job option show the collision; successful calls are unaffected because completed jobs are removed immediately.
Production impact at
3c7e57a: Webhook retry change cannot degrade the web serviceRepository lint:
pnpm run lint(declared in package.json) fails over the whole repository (the declared target takes no file arguments), so the failure is not attributed to this change and did not block it; run it before merging.Fix walkthrough
Watch the fix explained (1:33). Recorded for commit
3c7e57a1c8476962fa605d07d77a7580863d885d.Full analysis
Detection evidence
Affected:
int_13ee3c91c001c2c8xfdj06a9Out of scope
Production impact
The changed path is the web service's webhook producer (
apps/web/src/server/service/webhook-service.ts). The only new runtime effect is aQueue.remove(jobId)before the existingQueue.add; BullMQ's remove returns 0 without throwing for a missing or locked job, so no error path is added. I read the productionunsendRailway service (tier 2, serves app.usesend.com) and could not read live metrics because the Railway account returns 403 Not Authorized, so the read is empty.Affected resources
unsendObservability
[WebhookQueueService]: Worker errorlogs a failed BullMQ job; the changedenqueueCallis upstream of it.Webhook call <id> completed successfullyandwebhookCall.statustransitions (PENDING, IN_PROGRESS, DELIVERED, FAILED) show whether a retry was delivered.[WebhookQueueService]: Call not foundand the discarded-call log cover the non-active paths.Polylane assessed this pull request at
3c7e57a.Causal chain
2 files changed (+82/-1)
apps/web/src/server/service/webhook-service.ts: modified, +5/-0apps/web/src/server/service/webhook-service.unit.test.ts: modified, +77/-1Lint output
Generated by Polylane. You can ask follow-ups by mentioning @polylane in a comment. If you close this pull request without merging, add a comment that says why. Polylane reads it to make its next fixes better.
Summary by cubic
Retrying a failed webhook delivery returned success but sent nothing. BullMQ keeps failed jobs for 30 days under the call id, and
add()ignores a job id that already exists, so the retry never queued a new delivery. The retry now removes the retained job before enqueuing a fresh one, and completed deliveries are unaffected because those jobs are removed immediately.Queue.remove(callId)before the existingQueue.addinenqueueCall, with tests covering the retry and non-retry paths.Written for commit 3c7e57a. Summary will update on new commits.