Repository navigation
feat(stats): show how FTW Cloud runs in the private view - #105
Conversation
The "Energy via FTW Cloud" section shows the newest public day of the anonymous daily energy reports: its report count and its solar, grid, battery and vehicle sums, and the public days of the last 30. Labelled as reports, not homes. The Worker reads https://cloud.ftw.energy/v1/energy/public hourly with the other Cloud feeds, parses it as strictly as the fleet feed (exact fields, closed dates only, at least five reports or all null, sums in range) and keeps only the latest copy in D1 (migration 0004). A Cloud that does not serve the route yet is recorded as awaiting source, not as a failure. Pairs with srcfl/ftw and srcfl/ftw-cloud feat/anonymous-energy-ohttp. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
Deploying with
|
| Status | Name | Latest Commit | Preview URL | Updated (UTC) |
|---|---|---|---|---|
| ✅ Deployment successful! View logs |
ftw-webapp | 51c4d64 | Commit Preview URL Branch Preview URL |
Oct 24 2026, 09:42 AM |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c30e3b8d43
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| function maxOf(rows: { latency: Record<string, Latency> }[], key: string): number | null { | ||
| const v = rows.map(r => r.latency[key]?.p99).filter((p): p is number => typeof p === 'number') | ||
| return v.length ? Math.max(...v) : null |
There was a problem hiding this comment.
When an hour has samples but its p99 exceeds the feed's 10-second histogram limit, its p99 is null (which the dashboard's ms() formatter displays as “over 10 s”). Filtering out every null here makes a faster hour win instead: an hour with n: 1, p99: null followed by an hour with p99: 5 produces a daily slowest p99 of 5 ms. This hides the worst latency precisely during slowdowns. Distinguish absent or zero-sample series from overflowing quantiles and propagate the latter through the maximum; also ensure the assistant-call renderer does not suppress that result.
Useful? React with 👍 / 👎.
| alert: pushResults(totals, 'apns_alert_'), widgets: pushResults(totals, 'apns_widgets_'), browser: pushResults(totals, 'webpush_'), | ||
| breaker_opened: totals['apns_breaker_open'] ?? 0, coalesced: totals['push_coalesced'] ?? 0, | ||
| }, | ||
| mcp: { outcomes: prefixed(totals, 'mcp_'), p99_ms_max: maxOf(rows, 'mcp_call_seconds') }, |
There was a problem hiding this comment.
Once more than seven days of hours have been collected, this maximum includes the entire query window (at least 30 days), whereas the adjacent outcome counts include only the last seven days. For example, a 9-second p99 from ten days ago still appears under “Assistant calls, 7 days” even if every call this week completed in 5 ms. Filter the latency rows to the same seven-day window before calling maxOf so old incidents age out of the weekly panel.
Useful? React with 👍 / 👎.
…hanged Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
3b81c2b to
9e09a86
Compare
Review of #105: - A quantile past ftwcloudd's last bucket comes as null beside a count. It was dropped or read as 0; it now reads as "over 10 s" (over_10s in the API), the slowest case, in every list and in forward_p99_ms. - The slowest assistant call keeps to the 7 days the panel names. - The public page no longer carries the panel's markup, copy or script. A test reads the public HTML. Also: a week without hours says so instead of "Nothing dropped"; old daily totals read stale; missing hours are gaps in the chart, with their own tooltip; the shards of an hour are added together; data out says how many of the month's hours it covers. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… is stale From review of #104: - A feed stamped up to 2 h ahead of the Worker could pass a day as closed one day early. The closed-day bound now uses the earlier of the feed's stamp and the Worker's clock. - observed_at and each date must be strings; an array no longer passes through String(). - The daily energy feed has its own freshness entry ("Daily energy", stale after 48 hours, since Cloud adds a day once a day); the hourly usage dot is now "Hourly energy". A stale feed says so in the note and on the tiles, also when every day is withheld. - Cloud now withholds a sum that one to four of a day's reports had any of (srcfl/ftw-cloud#50). The parser takes null for such a sum, and the 30-day window sums each field over the days that show it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Worker reads FTW Cloud's internal feed (GET /v1/ops, with the CLOUD_OPS_KEY secret) once an hour at minute 23 and keeps it in D1: the service's hourly sums for 90 days, and the latest exact totals of closed fleet and energy report days. The parser takes only the fields ftw.cloud-ops/1 names, at every level, and refuses the whole answer otherwise; a test holds it to a feed ftwcloudd wrote. The private view shows boxes and phones connected, the share on QUIC, data written this month against AWS's 100 GB of free data out, drops by reason, push results, assistant calls by outcome, the slowest p99 of the day, each deploy's drain, and the exact daily reports. Nothing in it names a home, a device or an address. The public view is unchanged. Stacked on #104, whose energy fields it shares. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
FTW Cloud's /v1/ops carries the health counts of ftw.fleet/2 reports in its closed fleet days (srcfl/ftw-cloud#52, srcfl/ftw#1821): the slowest plan and fallback plans of each box's last day, the share of failed driver reads, and the newest update of its last week. The parser takes a day with all four, each from its fixed list and none above the day's reports, or a day with none, and refuses anything between. The private view shows them beside the other labels. The fixture is ftwcloudd's own answer with an ftw.fleet/2 report in it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review of #105: - A quantile past ftwcloudd's last bucket comes as null beside a count. It was dropped or read as 0; it now reads as "over 10 s" (over_10s in the API), the slowest case, in every list and in forward_p99_ms. - The slowest assistant call keeps to the 7 days the panel names. - The public page no longer carries the panel's markup, copy or script. A test reads the public HTML. Also: a week without hours says so instead of "Nothing dropped"; old daily totals read stale; missing hours are gaps in the chart, with their own tooltip; the shards of an hour are added together; data out says how many of the month's hours it covers. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The public energy feed may now show null for a sum few reports had (#104). The internal feed is exact and never does; its parser refuses a null sum. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
415b51f to
35c5cc9
Compare
# Conflicts: # stats/src/index.ts # stats/src/store.ts # stats/test/worker.test.ts
Problem
Fredrik, 24 October 2026: Sourceful needs internal statistics to run FTW Cloud and understand the fleet, for internal use, in one place, without building a separate system. The private view of stats.ftw.energy is that place: it already sits behind Cloudflare Access and checks the JWT itself.
What changes
stats/src/ops.ts, minute 23 each hour): readshttps://cloud.ftw.energy/v1/opswith the newCLOUD_OPS_KEYsecret, from an hour before the newest hour kept, so a restart's late part of an hour replaces the row. No key: the run recordsawaiting key. Cloud without a key (404):awaiting key on Cloud. A refusal, a redirect, an answer over 1 MiB or one bad field: the run fails and nothing is stored.parseOps), in the style ofstats/src/fleet.ts: exact field sets at every level (feed, hour, gauge, latency, fleet day, energy day), series names^[a-z0-9_]{1,80}$, whole non-negative counts, gauges with min ≤ mean ≤ max, quantiles in order and only with a count, settled hours only, days only once closed (a fleet day after it ends, an energy date four days after), at most a week of hours. A test runs it against a feedftwclouddwrote (stats/test/fixtures/cloud-ops.json).0005_cloud_ops.sql):cloud_ops_hours(90 days) andcloud_ops_days(the latest exact view only). Both pruned hourly.stats/README.md("FTW Cloud operations"), the secret, retention.Why none of it names a home
Cloud sends sums over the whole service and totals of reports that carry no id, and only closed days, so two pulls never differ by one report. The parser refuses any field the schema does not name, and a test checks the D1 columns. The limit, plainly: while the fleet is a handful of homes, a small count can still describe one of a few people staff know. srcfl/ftw-cloud
privacy/internal-operations.mdhas the full account.Health counts (second commit)
srcfl/ftw#1821 adds four coarse health buckets to the anonymous fleet report (
ftw.fleet/2), and srcfl/ftw-cloud#52 carries their exact counts in/v1/opsfleet days. The parser takes a day with all four health maps, each label from its fixed list and no count above the day's reports, or a day with none, and refuses anything between. The private view shows them as chips beside the other labels (reviewed in a browser). The public view does not show them. The fixture now holds anftw.fleet/2report asftwclouddwrites it.Review fixes (third commit)
over_10s, shown "over 10 s", the slowest case; it was dropped or read as 0.Pairs and order
energyFieldsit shares; its migration is0004, this one0005. Merge feat(stats): show FTW Cloud's anonymous daily energy reports #104 first, then rebase this ontomain./v1/ops), itself stacked on feat: one Oblivious HTTP gateway for both anonymous daily reports ftw-cloud#50, and for the health counts with feat(fleet): take ftw.fleet/2 and its health counts ftw-cloud#52 and feat(fleetping): say coarsely how the box ran in the anonymous daily report ftw#1821. The parser accepts days with and without health, so this can land before fix(now): keep the charger sheet off the first frame #52.Nothing is deployed. Turning it on: srcfl/ftw-cloud
deploy/README.md, "Watching it".Checks
npm run test:stats: 44 passed.npm run check:statsandnpm run build:statspass.npm run verify: four tests insrc/views/Energy.svelte.test.tsfailed under the full parallel run and pass alone (10/10); this PR does not touchsrc/.🤖 Generated with Claude Code