Skip to content

perf(telemetry-server): cache the stats snapshot for 60 seconds - #4092

Open
vavallee wants to merge 2 commits into
mainfrom
perf/telemetry-stats-cache
Open

vavallee wants to merge 2 commits into
mainfrom
perf/telemetry-stats-cache

Conversation

@vavallee

Copy link
Copy Markdown
Owner

Summary

/stats and /api/stats ran every dashboard query (a dozen aggregates over installs and the daily tables) on each request. They now share one cached *statsData for 60 seconds.

  • computeStats is now the cached entry point (stats_cache.go); the old body is renamed queryStats. Handlers are unchanged, which keeps this independent of fix(telemetry-server): stop logging cancelled stats requests as errors #4091.
  • TTL is server.statsTTL (60 s in main, zero means no cache, which is what newTestServer gets, so the existing tests still hit the DB every time).
  • Concurrent misses wait on one in-flight fill, hand rolled with a mutex and a done channel. golang.org/x/sync is not in go.mod, so no new module.
  • The fill runs on context.WithoutCancel(request) plus its own 30 s timeout. A visitor who leaves gets their context error back (which fix(telemetry-server): stop logging cancelled stats requests as errors #4091 then treats as client gone) and the fill carries on for everyone else and lands in the cache. A request whose context is already done does not start a fill.
  • Errors are not cached. A panic in the fill goroutine becomes an error for the waiters instead of killing the process (net/http used to recover it on the handler goroutine).
  • The snapshot is shared, so the count/label tie break sort moves from handleStatsPage into queryStats. Leaving it in the handler is a real data race: I put it back temporarily with the cache in place and -race flagged stats_page.go on concurrent renders. The renderers already copy before mutating.

Staleness: the dashboard footer now says figures are recomputed at most once a minute (it said "refreshes on every page load"), the endpoint list in main.go says cached up to 60 s, and deploy/README.md notes it. Nothing depends on these being live to the second: pings write straight to the DB, /stats.json (what discord-stats reads) is computed per request as before, tools/pingdrift reads only build_sha from /api/stats, which is the package var and not part of the cached data, and the backup workflow uses /api/backup.

Fragment: changelog.d/telemetry-stats-cache.md.

How it was verified

stats_cache_test.go, with a counting stub swapped in through server.statsQuery:

  • two calls within the TTL run the query once and return the same pointer; expiry lands at about the configured TTL
  • after the TTL (entry aged directly, plus a real 10 ms TTL) it reruns
  • 16 concurrent misses run it once and all get the same snapshot
  • an error is not cached; the next call reruns and succeeds
  • the starter cancels mid fill: it gets context.Canceled promptly, a waiter gets data, the fill context was never cancelled, a later request is served from cache, one query total
  • a panicking query returns an error and clears the in-flight marker
  • /stats, /api/stats, /stats run the query once
  • 8 concurrent /stats renders on a warm cache with tied buckets (race guard)
  • queryStats still applies the tie break

There is no fail before on main for the cache tests (the fields do not exist there), so here is mutation evidence instead:

fill on request ctx instead of WithoutCancel -> CancelledStarterDoesNotFailFill: fill context was cancelled; ran 2 times, want 1
no in-flight sharing                         -> ConcurrentMissesRunOnce: ran 16 times for 16 concurrent misses, want 1
expiry ignores statsTTL                      -> CachesWithinTTL, ShortTTLExpires fail
cache lookup disabled                        -> CachesWithinTTL, CancelledStarter..., ShareCache fail
sort left in handler, not in queryStats      -> ConcurrentRendersShareSnapshot: WARNING: DATA RACE stats_page.go

One mutation survives: caching on error too. A failed fill has nil data and the lookup requires non nil data, so it behaves the same.

go test -race ./cmd/telemetry-server/... green, golangci-lint 0 issues. Also merged this branch with #4091 locally: no conflict, and the combined package passes -race -count=2, including #4091's cancel tests through the cached path.

Checklist

  • Signed-off-by matches the author
  • Changelog fragment added
  • Tests added
  • No new dependency

Merging to main deploys: ping-server.yml pushes ghcr.io/vavallee/bindery-ping:sha-<commit>, and the infra repo's bump workflow (every 30 minutes) opens and auto merges a PR moving the pinned sha once that build is green.

🤖 Generated with Claude Code

/stats and /api/stats ran every dashboard query on each request. Cache the
computed statsData on the server for statsTTL (60 s in production, a field
so tests can shorten it), shared by both endpoints.

Concurrent misses wait on one in-flight fill instead of each running the
queries. The fill runs on context.WithoutCancel of the starting request
plus its own 30 s timeout, so the first visitor leaving does not fail it
for the others; a waiter whose own context ends gets its context error and
the fill still lands in the cache. Errors are not cached. A panic in the
fill goroutine becomes an error rather than taking the process down.

The snapshot is now shared, so the tie break sort moves from
handleStatsPage into queryStats (the old computeStats). Leaving it in the
handler is a data race under concurrent renders, which the new concurrent
render test catches with -race.

/stats.json and pings are untouched. The dashboard footer and
deploy/README.md note the up to 60 s staleness.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Signed-off-by: vavallee <vavallee@protonmail.com>
@codecov

codecov Bot commented Nov 20, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant