Repository navigation
fix(web): start the webhook cleanup job at server startup - #507
polylane[bot] wants to merge 1 commit into
Conversation
Co-authored-by: polylane[bot] <277585245+polylane[bot]@users.noreply.github.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
Important Review skippedBot user detected. To trigger a single review, invoke the
|
Deploying usesend with
|
| Latest commit: |
2ff11b3
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://0f7faa06.usesend.pages.dev |
| Branch Preview URL: | https://polylane-autofix-4l1lzj5n11m.usesend.pages.dev |
|
Thanks, confirmed: the job was never imported, and production logs show no run since webhooks launched. I'm replacing this with #521, because its first run would delete about 10 months of rows in one The replacement deletes in batches of 1,000, each with lock and statement timeouts and a 10-minute nightly budget. It only deletes finished calls not touched in 30 days, so a call someone just retried is kept. It's cloud only, so self-hosters don't lose history on upgrade. It adds an integration test against real Postgres. |
…in small batches (cloud) (#521) * fix(web): start the webhook cleanup job, and delete old calls in small batches The nightly webhook cleanup job was never imported, so it never ran: webhook call history has grown since webhooks shipped in January. - Start it at server startup with the other Redis-backed jobs. - Delete in batches of 1000 ids (each its own short transaction) with a short pause between batches, stopping after 10 minutes; the next run carries on. A single DELETE of ~10 months of rows would write a burst of WAL and hold a long transaction on a database whose disk is ~78% full. Supersedes #507. Co-authored-by: Polylane <polylane[bot]@users.noreply.github.com> * fix(web): only delete finished, untouched webhook calls, with per-batch timeouts, on cloud only - Delete only DELIVERED/FAILED/DISCARDED calls created and last updated over 30 days ago; the DELETE repeats these conditions, so a call retried after it was selected is kept. - Each batch runs in its own transaction with lock_timeout 5s and statement_timeout 60s; the deadline is checked before each batch; a failed batch stops the run, keeping what was already deleted. - Start the job on cloud only; self-hosted installs keep their history until there's a setting to opt in. - Log progress every 50 batches and the run's elapsed time; comments no longer claim the cleanup shrinks the database files. * fix(web): give each cleanup batch a Prisma transaction limit above its SQL timeouts; test against postgres Prisma's default interactive-transaction timeout (5s) would have cut a slow batch off before its 60s statement_timeout, possibly failing on the same batch every night. Adds an integration test that runs the cleanup against a real Postgres (CI): only finished calls created and last updated over 30 days ago are deleted. * test(web): register the cleanup-job mock per test, so the self-hosted check can fail --------- Co-authored-by: Polylane <polylane[bot]@users.noreply.github.com>
Webhook delivery history was never cleaned up, so the shared database grew with every delivery for every team. The server now starts the daily cleanup job that removes records older than the retention window, which bounds the table's size. The job registers its schedule when its module loads, and no code imported it, while every other background job starts in the same function. Adding that import starts the job. The retention window and the database schema are unchanged.
Production impact at
2ff11b3: Startup adds a daily cleanup job; no schema or request-path changeRepository lint:
pnpm run lint(declared in package.json) fails over the whole repository (the declared target takes no file arguments), so the failure is not attributed to this change and did not block it; run it before merging.Fix walkthrough
Watch the fix explained (1:23). Recorded for commit
2ff11b3cf3c56aca43337659d83d1c623833086e.Full analysis
Detection evidence
Affected:
int_13ee3c91c001c2c8xfdj06a9Out of scope
Production impact
I checked the deployed Railway service
unsend(production, 1 replica), which this repository deploys to. The changed function only starts background jobs at boot and is on no request path. The new job registers one BullMQ schedule and deletes rows older than 30 days once daily: additive, no schema change, removes nothing. Railway metrics for the service returned 403 Not Authorized with the read-only token, so its traffic was not readable; the change carries no production mechanism to degrade it.Affected resources
unsendObservability
Registering instrumentationinapps/web/src/instrumentation.ts#register.[WebhookCleanupJob]: Deleted old webhook callswith the deleted count and cutoff, plus[WebhookCleanupJob]: Job completed/Job failed.Polylane assessed this pull request at
2ff11b3.Causal chain
2 files changed (+65/-0)
apps/web/src/instrumentation.ts: modified, +2/-0apps/web/src/instrumentation.unit.test.ts: added, +63/-0Lint output
Generated by Polylane. You can ask follow-ups by mentioning @polylane in a comment. If you close this pull request without merging, add a comment that says why. Polylane reads it to make its next fixes better.
Summary by cubic
Starts the webhook cleanup job at server startup so webhook delivery history no longer grows unbounded in the shared database.
The cleanup job registers its daily scheduler and worker only when its module is imported, and nothing imported it before. This adds that import to the server boot sequence and adds unit tests covering both configured and missing Redis URLs. The 30-day retention window and schema are unchanged.
Written for commit 2ff11b3. Summary will update on new commits.