Repository navigation
Analytics: deploy evidence, record endpoints and bounded queries - #263
Merged
Merged
Conversation
…number
Analytics counted build rows and inferred outcomes from status columns that any run could
overwrite. Nothing showed which records a number counted, and long windows cost the database
seconds per request. The deploy pipeline now records its own evidence, every number opens its
records, and the queries are bounded.
Migration 038 adds the build_events table and the first-ready, first-rollout, last-teardown and
namespace stamps on builds. It adds the analytics_retention setting, three small build_events
indexes and a deploys("buildId") index unless one exists. It resumes after an interrupted run:
finished indexes are kept, an invalid one is rebuilt, and the builds backfills run in id batches.
The reconciliation worker records every deploy request with its origin, every run claim, every
block and every exit that stops a run. Each finish records its outcome, services, per-service
results and failure reason. A request resolves by its own run id. The last finish that followed a
rollout decides. A takeover counts as superseded. A service that fails before it has a row fails
the run. A teardown that stopped a run, or preceded it, counts as torn down.
New endpoints summarize deploy requests and MCP tool calls and list the records behind each
number. deployments/requests and mcp/calls reuse the summary queries, cut by a slice. A slice is
an outcome, a source, a failure reason, a tool, a user, a client application, one chart bucket or
the previous period. A live-database test proves every slice total matches its summary count.
environments/records accepts the evidence levels and a chart bucket. environments/pull-requests
lists the PR records behind PRs recorded and PRs deployed. Agent runs and sessions accept a chart
bucket and the in_progress status group. Environment analytics report evidence levels (recorded,
deploy requested, rolled out, live, ready), exclude the seed record and link every record that
still has a page. Lifetimes tell recorded end times from estimates and measure runtime from the
first deploy. MCP audits every tool call with its client application, and a daily job applies
the retention setting. Telemetry stats use the shared calendar window, and agent cost estimates
price cached tokens.
Every analytics transaction runs with JIT off, at most four at once with a short queue, and
record lists scan only their slice. Agent queries decompress each run plan once.
Gated live-database and in-memory worker suites cover the attribution rules, and the OpenAPI
schemas describe every field the UI reads.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Analytics counted build rows and inferred outcomes from status columns that any run could overwrite. Nothing showed which records a number counted, and long windows cost the database seconds per request.
What changes
build_events, the first-ready, first-rollout, last-teardown and namespace stamps onbuilds, theanalytics_retentionsetting, three smallbuild_eventsindexes and adeploys("buildId")index unless one exists. The worker records every request, claim, block, exit and finish with its outcome, services, per-service results and failure reason. A request resolves by its own run id.deployments/requestsandmcp/callsreuse the summary queries cut by a slice, and a live-database test proves every slice total matches its summary. Environment analytics report evidence levels and link every record that still has a page. Lifetimes tell recorded end times from estimates and measure runtime. A daily job applies the retention setting.Rollout notes
pnpm db:migrateonce from a one-off pod before deploying, or raise the probe for this release. The migration resumes after an interruption: finished indexes are kept, an invalid one is rebuilt, and thebuildsbackfills run in id batches of 5,000. A killed run leaves the knex lock set; clear it withknex migrate:unlock.timezone=UTCanddatestyle=ISO,MDYthrough theoptionsstartup parameter. CheckSHOW timezoneon the target. If it is not UTC, rows written before this release keep the old offset. PgBouncer older than 1.20 needsignore_startup_parameters = options.analytics_retentionkeeps MCP tool calls and build events forever by default. SetmcpToolCallDaysandbuildEventDaysinglobal_configto prune.Validation
10,000 tests with coverage at 98.12/91.86/98.12/98.52, the gated live-database suite, the server tsconfig, and the migration exercised fresh and resumed on a copy of a populated database.
pnpm ts-checkcarries pre-existing errors in unrelated files; this branch adds none.Pairs with https://github.com/GoodRxOSS/lifecycle-ui/pull/93.