Skip to content

fix: make the admin stats endpoint about 3x faster - #59

Merged
KMKoushik merged 1 commit into
mainfrom
fix/admin-stats-speed
Nov 17, 2026
Merged

KMKoushik merged 1 commit into
mainfrom
fix/admin-stats-speed

Conversation

@KMKoushik

Copy link
Copy Markdown
Contributor

On prod, GET /api/admin/stats takes 3.5–5 s, and days=90 hits the 5 s statement timeout. This brings it down by about 3×. It makes no schema changes, adds no indexes and keeps the same response shape. On a prod-sized seed the output is byte-identical to before for days = 1, 7, 30, 90 and 365.

Why it was slow

I measured with EXPLAIN (ANALYZE, BUFFERS) on a fictional seed the size of prod (27k users, 12.9k groups, 245k expenses, 857k participant rows, 223 MB), running on Postgres 16 in a container limited to 1 CPU and 1 GB RAM, with the default work_mem of 4 MB.

  • Active-users query: ~1.1 s of the ~1.2 s total. The 6-month weekly and monthly windows are always read, which is why days=7 was already slow.
    • It built 431k (user, time) rows without removing duplicates first.
    • It hashed all 857k ExpenseParticipant rows. That spilled to disk in 16 batches.
    • It ran three COUNT(DISTINCT) sorts on to_char text keys. All three were external merge sorts on disk, about 600 ms together.
  • Date bounds bound as text: each date bound was sent as a text parameter cast to timestamp. Postgres re-parsed it for every row, which made each Expense scan about 4× slower (~57 ms instead of ~13 ms).
  • JIT: compiling the large query cost about 140 ms.

Fix

  • Dedupe before counting: active users are first reduced to one row per (day, user), using a date column and an in-memory hash aggregate. Weeks and months are counted from those rows, so the 6-month window is read and grouped once.
  • Better join: the participant join now hashes only the expenses in the window and streams ExpenseParticipant past them.
  • Signed-in check after dedup: whether a user has signed in is checked once the rows are deduplicated, as a hashed semi-join.
  • Date bounds as literals: they're now inlined as typed literals. The text only ever comes from Date.toISOString() and is validated before it's inlined.
  • This transaction only: SET LOCAL work_mem = '32MB', so nothing spills to disk, and SET LOCAL jit = off.
  • No indexes:
    • An Expense(createdAt) index would build in 49 ms but saves only 10–20 ms, because the monthly window always covers about 40% of expenses.
    • ExpenseParticipant(expenseId) is already covered by the primary key.
    • The statement timeout stays at 5 s.

Timings (fictional prod-sized seed, median of runs after the first)

days=7 days=30 days=90 days=365
Before, 1 CPU 1.10 s 1.17 s 1.35 s 1.79 s
After, 1 CPU 0.38 s 0.39 s 0.43 s 0.54 s
Before, 0.5 CPU 2.41 s 2.45 s 3.01 s 3.70 s
After, 0.5 CPU 0.82 s 0.85 s 0.90 s 1.15 s

The 0.5-CPU "before" numbers are close to what prod shows. The active-users query went from 1,103 ms to about 340 ms, and the daily expense counts from 67 ms to 25 ms. No disk spills remain.

Tests

  • tsc, next lint and prettier pass.
  • The full suite passes 362/362 serially, and in one of two parallel runs.
  • The other parallel run hit the known flakes that also happen on main: three unrelated tests failed (the cent-distribution, native push and saved-expense tests).

Count active users from one row per user and day (hash aggregates in memory)
instead of three COUNT(DISTINCT) sorts over every involvement row, join
participants by hashing only the expenses in the window, inline the date bounds
as typed literals (a bound text param cast to timestamp is re-parsed per row),
and run with work_mem 32MB and JIT off. Same response; no schema changes.
@coderabbitai

coderabbitai Bot commented Nov 16, 2026

Copy link
Copy Markdown

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 24 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 72731772-503a-46f2-af2b-adc8e9621dc3
Commits

Reviewing files that changed from the base of the PR and between a50bd86 and 0353015.

Files selected for processing: 1

  • src/server/api/services/adminStats.ts
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@KMKoushik
KMKoushik merged commit ce18c77 into main Nov 17, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant