Skip to content

feat(fleetping): say coarsely how the box ran in the anonymous daily report - #1821

Merged
frahlg merged 9 commits into
masterfrom
feat/fleet-report-health
Oct 24, 2026
Merged

frahlg merged 9 commits into
masterfrom
feat/fleet-report-health

Conversation

@frahlg

@frahlg frahlg commented Oct 24, 2026 •

Copy link
Copy Markdown
Member

Problem

Fredrik, 24 October 2026: Sourceful needs to understand the fleet's operation, internally and in aggregate, to know where engineering time goes. The anonymous daily report says what a box is (version, drivers, battery, zone, age), not how it runs: is the planner fast enough on the hardware boxes use, does it fall back, do drivers fail, do updates fail.

What changes

ftw.fleet/2 adds four buckets to the same report, under the same switch (projectReportsAllowed: the saved switch and the project statistics answer):

Field Buckets Source in Core
plan_solve none, 0-1s, 1-5s, 5s+: the slowest plan of the last 24 h planner_diagnostics (kept 7 days), new state.PlannerDay
plan_fallbacks none, 0, 1-3, 4+: plans of the last 24 h that fell back the same query, $.solver.fallback
driver_errors none, 0, 0-1%, 1%+: share of driver polls that failed in the last 24 h new 24-slot hourly count in the telemetry store (CountPoll from the driver registry's poll loop, PollsLastDay), all drivers together
update none, ok, failed: the newest update of the last 7 days selfupdate.Checker.Status(), action update only
  • fleetwire.Decode takes ftw.fleet/1 (7 fields) and ftw.fleet/2 (11 fields, each health value from its fixed list) and nothing else. A Day counts health only from reports that carry it; a day without one has no health fields, so the public view and older readers see what they saw.
  • Settings → FTW app → Anonymous daily report names the new fields in its paragraph ("ten things") and shows them in the preview, in plain words. Reviewed in a browser against a local Core.
  • docs/architecture.md, "Fleet ping", says what the buckets are and where they come from.
  • Rebased on feat(stats): one statistics answer, and anonymous daily reports through an OHTTP relay #1820 at a7c025e (report id and day as inner request fields), on top of its relay rework (4cfd65f): the fleet report now travels through the relay inside one padded inner request. reportwire's TestEveryReportHasOneLength now builds the largest ftw.fleet/2 report, with the longest health values. Its body is 1,331 bytes, and with the report id and day the padded request still fits 1,536.

How driver_errors counts (review fixes)

  • Only the driver registry's poll loop counts, a failed poll and a good one alike (CountPoll). An OCPP charger's messages record health but are not polls and are counted neither way. Before, an OCPP-only site with one faulted connector read 1%+ (TestOnlyPollsAreCounted, TestRunLoopCountsPollsAndFailedPolls).
  • The count lives in memory. Until it covers six hours, as after an update's restart, the box reports none ("too few reads to say" in Settings), so a start-up burst of failed polls cannot read as trouble. none therefore means no polls or too few hours of them.

Why these fields cannot single out a box

  • Few values, a common answer. Each field has at most four values (a test holds that), and most boxes share one: 0 fallbacks, 0 errors, none updates. A rare answer sits in a crowd of boxes that differ in it and agree on everything else.
  • They describe the last day or week, not the house. They move as the box runs, so they are a weaker join key between two days' reports than the static fields the report already carries. They add no counter, no time and no id: the slowest solve is a bucket, never a duration; errors are a share, never a count; an update is an outcome, never a version.
  • No new stable trait. A box that does not update itself (Docker, Home Assistant) says none, like a native box that had no update, so the install method does not become a field. Driver errors are summed over all drivers, never named.
  • Only marginals survive. FTW Cloud keeps each field's daily count apart (fleetwire.Day), never the combination of fields, and keeps no report. The combination exists only in the request.
  • The relay hides the address. Since feat(stats): one statistics answer, and anonymous daily reports through an OHTTP relay #1820 the report reaches FTW Cloud through the relay, so the request no longer shows the box's address either.
  • Same honest limit as before. On a day with one report, the day's counts are that report. The quasi-identifier argument in go/internal/fleetping and on the Settings screen still holds; four coarse, changing fields do not change its shape.

Order and pairs

Checks

make verify passes (Go and Python suites, vet, build, protocol boundary). node --test web/fleet-ping-tab.test.mjs: 27 pass. New tests: TestHealthBucketsAreAFewFixedWords, TestADayCountsHealthOnlyFromReportsThatCarryIt (fleetwire), TestHealthTravelsOnlyAsBuckets (fleetping), TestPlannerDaySumsThePlansSinceATime (state), TestPollsLastDayCountsADayOfPollsAndErrors (telemetry), TestFleetHealthReadsTheBoxsLastDay (cmd/ftw).

The planner_diagnostics query runs json_extract over about a day of plan snapshots (roughly 100 rows of 85–170 kB) once a day and on each Settings preview. Not measured on a Pi.

🤖 Generated with Claude Code

…elay

Once per closed UTC day, 1 to 12 hours after it closes, the box sends the
date and six whole-kWh sums (solar, grid import and export, battery charge
and discharge, vehicle charge), sealed with Oblivious HTTP (RFC 9458) to FTW
Cloud's pinned gateway key and posted to an independent relay. The relay
never sees the report and FTW Cloud never sees the box's address.

- go/protocol/ohttp: RFC 9458 on crypto/hpke for one suite, with known-length
  Binary HTTP; the RFC's Appendix A vectors pass byte for byte.
- go/protocol/energywire: the report and its strict decoder.
- go/internal/energyreport: the complete-day rule on the energy ledger, the
  schedule and the sender. The relay URL and the gateway key are
  compile-time pins, both empty, so this release sends nothing.
- The fleet report's answer is the gate, shared in one function; a site a
  partner controls is left out. Settings shows the report beside the fleet
  report.
- ADR 0008 records what each party sees, the limits and the open ePrivacy
  question.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 24, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-24T06:34:52.953124Z 55bb9e4 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 55bb9e4849

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread go/internal/telemetry/store.go Outdated
s.mu.Lock()
defer s.mu.Unlock()
s.driverHealthLocked(name).RecordError(err)
s.countPollLocked(time.Now(), true)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Count failed polls separately from general driver errors

For an approved OCPP 2.0.1 charger, a faulted StatusNotification calls telError in go/internal/ocpp/handlers_v201.go, which reaches RecordDriverError without any poll occurring. This change counts that notification as a failed poll, while successful OCPP messages use RecordDriverSuccess and never enter the denominator. Consequently, an OCPP-only site with one connector fault reports a 100% poll failure rate regardless of its successful traffic, and mixed sites can also land in the wrong fleet-health bucket. Record poll outcomes explicitly in the registry's poll branch rather than treating every driver-health error as a failed poll.

Useful? React with 👍 / 👎.

…e relay

Fredrik's change of scope, 24 October 2026: statistics are one answer on
the box, and every report goes the same way.

- One answer for both anonymous reports (cloudlink.SharesStatistics): part
  of FTW Cloud under the current terms, with no off switch; on for a box
  without Cloud until its owner turns it off. Nobody is asked. Stored state
  is read once (MigrateStatistics): a local No, a fleet report switched off
  in the config, or Cloud turned off on an earlier release stays off; a
  local yes stays yes; every other box is on by default. fleet_ping.enabled
  is no longer a switch. A box without Cloud no longer starts a
  statistics-only Cloud link.
- A box that shares only by default posts nothing straight to FTW Cloud,
  which would see its address; it waits for the relay.
- go/internal/statsrelay holds the relay and gateway pins and sends sealed
  reports; the fleet report goes through it once a release names a relay,
  the energy report always. Each report travels alone at its own time.
- go/protocol/reportwire: one padded inner request for both kinds
  (1,536 bytes); the gateway picks the decoder by schema.
- Settings: Anonymous statistics with the one switch beside both messages;
  the Cloud panel no longer asks local boxes.
- ADR 0008 and the protocol doc record the model, the two-report rule and
  the open ePrivacy question.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
frahlg added a commit to srcfl/ftw-webapp that referenced this pull request Oct 24, 2026
FTW Cloud's /v1/ops carries the health counts of ftw.fleet/2 reports in
its closed fleet days (srcfl/ftw-cloud#52, srcfl/ftw#1821): the slowest
plan and fallback plans of each box's last day, the share of failed
driver reads, and the newest update of its last week. The parser takes a
day with all four, each from its fixed list and none above the day's
reports, or a day with none, and refuses anything between. The private
view shows them beside the other labels. The fixture is ftwcloudd's own
answer with an ftw.fleet/2 report in it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@frahlg
frahlg force-pushed the feat/fleet-report-health branch from 55bb9e4 to da5077f Compare October 24, 2026 06:43
frahlg added a commit to srcfl/ftw-cloud that referenced this pull request Oct 24, 2026
Moves the protocol pin to srcfl/ftw#1821, whose fleet report adds four
health buckets: the slowest plan of the box's last day, how many plans
fell back, the share of driver polls that failed, and the newest update
of its last week. Both routes, /v1/fleet and the relay's /v1/ohttp,
take ftw.fleet/1 and ftw.fleet/2; the gateway dispatches both schemas
to the fleet totals, so a box released between #1820 and #1821 still
counts. One daily count per field, never their combination.
/v1/fleet/public leaves the health counts out; /v1/ops carries them,
exact, for closed days.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
frahlg added a commit to srcfl/ftw-webapp that referenced this pull request Oct 24, 2026
FTW Cloud's /v1/ops carries the health counts of ftw.fleet/2 reports in
its closed fleet days (srcfl/ftw-cloud#52, srcfl/ftw#1821): the slowest
plan and fallback plans of each box's last day, the share of failed
driver reads, and the newest update of its last week. The parser takes a
day with all four, each from its fixed list and none above the day's
reports, or a day with none, and refuses anything between. The private
view shows them beside the other labels. The fixture is ftwcloudd's own
answer with an ftw.fleet/2 report in it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
frahlg added a commit to srcfl/ftw-cloud that referenced this pull request Oct 24, 2026
Moves the protocol pin to srcfl/ftw#1821, whose fleet report adds four
health buckets: the slowest plan of the box's last day, how many plans
fell back, the share of driver polls that failed, and the newest update
of its last week. Both routes, /v1/fleet and the relay's /v1/ohttp,
take ftw.fleet/1 and ftw.fleet/2; the gateway dispatches both schemas
to the fleet totals, so a box released between #1820 and #1821 still
counts. One daily count per field, never their combination.
/v1/fleet/public leaves the health counts out; /v1/ops carries them,
exact, for closed days.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
frahlg added a commit to srcfl/ftw-webapp that referenced this pull request Oct 24, 2026
FTW Cloud's /v1/ops carries the health counts of ftw.fleet/2 reports in
its closed fleet days (srcfl/ftw-cloud#52, srcfl/ftw#1821): the slowest
plan and fallback plans of each box's last day, the share of failed
driver reads, and the newest update of its last week. The parser takes a
day with all four, each from its fixed list and none above the day's
reports, or a day with none, and refuses anything between. The private
view shows them beside the other labels. The fixture is ftwcloudd's own
answer with an ftw.fleet/2 report in it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
From review of srcfl/ftw-cloud#50: the gateway counted a replayed request
again, and a fleet report, which has no date, could be replayed on any later
day. Each inner request now carries a random report id and the UTC date it
was sent, as Binary HTTP header fields inside the sealed message, at the
same fixed length. reportwire.Fresh refuses a report sent before yesterday;
the gateway keeps an id hash per date to count each report once.

ADR 0008 also states the open limit: the floor of five counts reports, and
anyone with the key configuration can post them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
frahlg added a commit that referenced this pull request Oct 24, 2026
…rs six hours

Review of #1821: OCPP errors fed the poll count through
RecordDriverError while OCPP successes did not, so an OCPP-only site
with one faulted connector read 1%+. Only the driver registry's poll
loop counts now (telemetry CountPoll), a failed poll and a good one
alike; health records are not polls.

The count lives in memory, so after an update's restart it covered
minutes and a start-up burst of failed polls could read as trouble.
PollsLastDay now says how much of the day it covers, and the report
says none until that is six hours.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@frahlg
frahlg force-pushed the feat/fleet-report-health branch from da5077f to 45a3e55 Compare October 24, 2026 07:15
frahlg added a commit to srcfl/ftw-cloud that referenced this pull request Oct 24, 2026
Moves the protocol pin to srcfl/ftw#1821, whose fleet report adds four
health buckets: the slowest plan of the box's last day, how many plans
fell back, the share of driver polls that failed, and the newest update
of its last week. Both routes, /v1/fleet and the relay's /v1/ohttp,
take ftw.fleet/1 and ftw.fleet/2; the gateway dispatches both schemas
to the fleet totals, so a box released between #1820 and #1821 still
counts. One daily count per field, never their combination.
/v1/fleet/public leaves the health counts out; /v1/ops carries them,
exact, for closed days.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… new sessions

From review of #1820:
- The hourly C8 upload read only the grant's stats purpose. It now reads
  the one answer too: off, or Cloud under older terms, sends no hour. Under
  older terms, asking for statistics is refused (ErrCloudTermsFirst, 409)
  instead of adding the stats purpose beside the old terms; Status says so
  (statisticsNeedTerms) and Settings shows it. After the migration,
  ReconcileStatistics takes the stats purpose out of a grant whose answer is
  off, and ends a statistics-only link, as turning statistics off does.
- A stored grant counts only through the running link, which honours it
  only with its key (Link.SharesStatistics, Link.SaidYesToStatistics). With
  no link, the box's own answer alone.
- While a Cloud that was turned off waits for its deletion receipt, the
  local answer can still change.
- energyreport.Day: a charger's session counter starts again at every
  plug-in, which the ledger marks as a reset and covers from power; that no
  longer drops the day. An asset first seen after the day ended is not part
  of it.
- statsrelay: FTW Cloud's host is refused in any case and with a trailing
  dot.
- ADR 0008 records Fredrik's decisions: a default box waits for the relay,
  and the floor of five counts reports, accepted, with Privacy Pass later.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
frahlg added a commit that referenced this pull request Oct 24, 2026
…rs six hours

Review of #1821: OCPP errors fed the poll count through
RecordDriverError while OCPP successes did not, so an OCPP-only site
with one faulted connector read 1%+. Only the driver registry's poll
loop counts now (telemetry CountPoll), a failed poll and a good one
alike; health records are not polls.

The count lives in memory, so after an update's restart it covered
minutes and a start-up burst of failed polls could read as trouble.
PollsLastDay now says how much of the day it covers, and the report
says none until that is six hours.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@frahlg
frahlg force-pushed the feat/fleet-report-health branch from 45a3e55 to 84da122 Compare October 24, 2026 07:49
frahlg added a commit to srcfl/ftw-cloud that referenced this pull request Oct 24, 2026
Moves the protocol pin to srcfl/ftw#1821, whose fleet report adds four
health buckets: the slowest plan of the box's last day, how many plans
fell back, the share of driver polls that failed, and the newest update
of its last week. Both routes, /v1/fleet and the relay's /v1/ohttp,
take ftw.fleet/1 and ftw.fleet/2; the gateway dispatches both schemas
to the fleet totals, so a box released between #1820 and #1821 still
counts. One daily count per field, never their combination.
/v1/fleet/public leaves the health counts out; /v1/ops carries them,
exact, for closed days.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
frahlg added a commit to srcfl/ftw-cloud that referenced this pull request Oct 24, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
frahlg and others added 3 commits October 24, 2026 10:09
The settings text and docs now say that a report sent through the relay
carries two sealed things beside it: a random number drawn new for each
report, and the UTC day it was sent. Settings shows both under each
report.

The hourly C8 upload now needs a given yes (Cloud under the current terms
or a local yes). A box that shares only by default sends no hour.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…report

The fleet report (ftw.fleet/2) adds four buckets: the slowest plan of the
last 24 hours, how many of those plans fell back, the share of driver
polls that failed, and the newest update of the last 7 days. Core reads
them from the plans it keeps (planner_diagnostics, PlannerDay), a new
24-hour count of polls in the telemetry store (PollsLastDay), and the
updater's status. Each bucket is at most four words wide and describes
the last day or week, not the house. A box that does not update itself
says none, so the install method does not become a field.

fleetwire accepts ftw.fleet/1 and ftw.fleet/2, and a day counts health
only from reports that carry it. Same switch as the rest of the report;
Settings names the new fields and shows them in the preview.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…rs six hours

Review of #1821: OCPP errors fed the poll count through
RecordDriverError while OCPP successes did not, so an OCPP-only site
with one faulted connector read 1%+. Only the driver registry's poll
loop counts now (telemetry CountPoll), a failed poll and a good one
alike; health records are not polls.

The count lives in memory, so after an update's restart it covered
minutes and a start-up burst of failed polls could read as trouble.
PollsLastDay now says how much of the day it covers, and the report
says none until that is six hours.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@frahlg
frahlg force-pushed the feat/fleet-report-health branch from 84da122 to 6a04673 Compare October 24, 2026 08:14
frahlg added a commit to srcfl/ftw-cloud that referenced this pull request Oct 24, 2026
Moves the protocol pin to srcfl/ftw#1821, whose fleet report adds four
health buckets: the slowest plan of the box's last day, how many plans
fell back, the share of driver polls that failed, and the newest update
of its last week. Both routes, /v1/fleet and the relay's /v1/ohttp,
take ftw.fleet/1 and ftw.fleet/2; the gateway dispatches both schemas
to the fleet totals, so a box released between #1820 and #1821 still
counts. One daily count per field, never their combination.
/v1/fleet/public leaves the health counts out; /v1/ops carries them,
exact, for closed days.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
frahlg added a commit to srcfl/ftw-cloud that referenced this pull request Oct 24, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@frahlg
frahlg changed the base branch from feat/anonymous-energy-ohttp to master October 24, 2026 08:23
…ealth

# Conflicts:
#	docs/architecture.md
#	go/cmd/ftw/fleet_ping.go
#	go/cmd/ftw/fleet_ping_test.go
#	go/cmd/ftw/main.go
#	go/internal/fleetping/fleetping_test.go
#	go/protocol/reportwire/report_test.go
#	web/settings/tabs/fleet.js
frahlg added a commit to srcfl/ftw-cloud that referenced this pull request Oct 24, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@frahlg
frahlg merged commit 2d7c8eb into master Oct 24, 2026
23 checks passed
frahlg added a commit to srcfl/ftw-webapp that referenced this pull request Oct 24, 2026
* feat(stats): show FTW Cloud's anonymous daily energy reports

The "Energy via FTW Cloud" section shows the newest public day of the
anonymous daily energy reports: its report count and its solar, grid,
battery and vehicle sums, and the public days of the last 30. Labelled as
reports, not homes.

The Worker reads https://cloud.ftw.energy/v1/energy/public hourly with the
other Cloud feeds, parses it as strictly as the fleet feed (exact fields,
closed dates only, at least five reports or all null, sums in range) and
keeps only the latest copy in D1 (migration 0004). A Cloud that does not
serve the route yet is recorded as awaiting source, not as a failure.

Pairs with srcfl/ftw and srcfl/ftw-cloud feat/anonymous-energy-ohttp.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(stats): the fleet report takes the same relay, with its feed unchanged

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(stats): read the energy feed on the earlier clock and say when it is stale

From review of #104:
- A feed stamped up to 2 h ahead of the Worker could pass a day as closed
  one day early. The closed-day bound now uses the earlier of the feed's
  stamp and the Worker's clock.
- observed_at and each date must be strings; an array no longer passes
  through String().
- The daily energy feed has its own freshness entry ("Daily energy", stale
  after 48 hours, since Cloud adds a day once a day); the hourly usage dot
  is now "Hourly energy". A stale feed says so in the note and on the tiles,
  also when every day is withheld.
- Cloud now withholds a sum that one to four of a day's reports had any of
  (srcfl/ftw-cloud#50). The parser takes null for such a sum, and the 30-day
  window sums each field over the days that show it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(stats): show how FTW Cloud runs in the private view

The Worker reads FTW Cloud's internal feed (GET /v1/ops, with the
CLOUD_OPS_KEY secret) once an hour at minute 23 and keeps it in D1:
the service's hourly sums for 90 days, and the latest exact totals of
closed fleet and energy report days. The parser takes only the fields
ftw.cloud-ops/1 names, at every level, and refuses the whole answer
otherwise; a test holds it to a feed ftwcloudd wrote.

The private view shows boxes and phones connected, the share on QUIC,
data written this month against AWS's 100 GB of free data out, drops
by reason, push results, assistant calls by outcome, the slowest p99 of
the day, each deploy's drain, and the exact daily reports. Nothing in it
names a home, a device or an address. The public view is unchanged.

Stacked on #104, whose energy fields it shares.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(stats): show the fleet report's health counts in the private view

FTW Cloud's /v1/ops carries the health counts of ftw.fleet/2 reports in
its closed fleet days (srcfl/ftw-cloud#52, srcfl/ftw#1821): the slowest
plan and fallback plans of each box's last day, the share of failed
driver reads, and the newest update of its last week. The parser takes a
day with all four, each from its fixed list and none above the day's
reports, or a day with none, and refuses anything between. The private
view shows them beside the other labels. The fixture is ftwcloudd's own
answer with an ftw.fleet/2 report in it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(stats): read the slowest quantile right and keep the panel private

Review of #105:
- A quantile past ftwcloudd's last bucket comes as null beside a count.
  It was dropped or read as 0; it now reads as "over 10 s" (over_10s in
  the API), the slowest case, in every list and in forward_p99_ms.
- The slowest assistant call keeps to the 7 days the panel names.
- The public page no longer carries the panel's markup, copy or script.
  A test reads the public HTML.

Also: a week without hours says so instead of "Nothing dropped"; old
daily totals read stale; missing hours are gaps in the chart, with their
own tooltip; the shards of an hour are added together; data out says
how many of the month's hours it covers.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(stats): the exact feed has no withheld energy sum

The public energy feed may now show null for a sum few reports had
(#104). The internal feed is exact and never does; its parser refuses a
null sum.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
frahlg added a commit to srcfl/ftw-cloud that referenced this pull request Oct 24, 2026
* feat: an Oblivious HTTP gateway for anonymous daily energy reports

ftwcloudd is the gateway (RFC 9458) for the box's daily energy report,
which reaches it through an independent relay, so the service sees the
report and never the box's address.

- POST /v1/ohttp opens a request with the gateway key, takes only the one
  padded inner request with one valid report (energywire), adds its six
  sums to the date's row and answers a sealed 204 or 400. Nothing is kept
  per report: no row, logged body or address. One shared rate budget, and
  at most 100,000 reports a date.
- GET /.well-known/ohttp-gateway serves the key configuration.
- GET /v1/energy/public lists the closed dates of the last 30 days; a date
  with fewer than 5 reports shows null.
- The key comes from Parameter Store into tmpfs at boot, as the Noise key
  does (deploy/ohttp.md, deploy/ohttp.sh); keygen -ohttp prints the pin
  Core needs. Without a key the gateway routes answer 501.

Pairs with srcfl/ftw feat/anonymous-energy-ohttp; go/protocol is pinned at
its commit 28fddd802242.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat: one gateway route for both anonymous daily reports

Fredrik's change of scope, 24 October 2026: the fleet report moves onto
the same relay and gateway as the energy report, and the two stay apart.

- internal/energy becomes internal/gateway. POST /v1/ohttp reads the one
  padded inner request (go/protocol/reportwire), picks the decoder by
  schema, and adds a fleet report to the existing fleet_days totals that
  /v1/fleet/public reads and an energy report to energy_days. Nothing is
  kept per report.
- POST /v1/fleet stays for boxes on a Core release without a relay; it can
  go one stable Core release after the first that pins the relay.
- The request cap fits the 1,591-byte sealed request; README and
  deploy/ohttp.md describe both reports and the two-per-day traffic.
- go/protocol pinned at srcfl/ftw 4cfd65f7.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(gateway): withhold an energy sum fewer than five reports had any of

A date with five reports and one car published that car's whole charge in
vehicle_charge_kwh, because the sum counted the other four reports' zeros.
energy_days now counts, for each sum, the reports that had any of it, and
/v1/energy/public shows a sum only when none or at least five had some.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(gateway): count each report once, budget per kind, close the fleet view

From review of #50:
- Replays: each inner request now carries a random report id and its send
  day (srcfl/ftw a7c025e8). The gateway refuses a request sent before
  yesterday and keeps a hash of each id per day for three days, in a WITHOUT
  ROWID table with no time, so a replay or a retry adds nothing and is still
  answered 204. Seven posts of one ciphertext now count once.
- /v1/fleet/public lists closed UTC days only, so the relay cannot poll it
  and match a new count to the report it just forwarded.
- Each report kind has its own budget, spent only by a report that opened
  and decoded; junk no longer silences real reports.
- A 10-second deadline on reading the body of /v1/ohttp and /v1/fleet; a
  failed read closes the connection.
- ohttp keys: data after the last whole PEM block, such as a second key cut
  short in a rotation, is refused instead of silently ignored.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs: the floor of five counts reports, accepted; tokens later

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Answer a repeat before the budget, refuse any partial key block

The gateway now looks a report id up, read-only, before it spends the
kind's budget. A repeat gets a sealed 204 and costs nothing, so one
captured request sent many times cannot crowd out real reports.

The OHTTP key file must hold whole PKCS #8 blocks and nothing else. A
block cut short at either end, or text between blocks, stops the start.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(fleet): take ftw.fleet/2 and its health counts

Moves the protocol pin to srcfl/ftw#1821, whose fleet report adds four
health buckets: the slowest plan of the box's last day, how many plans
fell back, the share of driver polls that failed, and the newest update
of its last week. Both routes, /v1/fleet and the relay's /v1/ohttp,
take ftw.fleet/1 and ftw.fleet/2; the gateway dispatches both schemas
to the fleet totals, so a box released between #1820 and #1821 still
counts. One daily count per field, never their combination.
/v1/fleet/public leaves the health counts out; /v1/ops carries them,
exact, for closed days.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat: keep the service's hours for 90 days and serve them to the stats Worker

ftwcloudd now keeps an hourly summary of every series it counts in the
registry (ops_hours), for 90 days: each counter's rise, each gauge's low,
mean and high, each latency's buckets, how much of the hour a process ran
and how many started. A deploy hands over its part of the hour after the
drain; the next process adds the rest.

GET /v1/ops serves a week of settled hours at a time, with the exact
totals of the anonymous fleet and energy reports for closed days, only to
the key the project stats Worker holds. The service holds the key's
SHA-256 (-ops-key-sha256); without it the route answers 404 to everyone.

New series: sockets by transport (QUIC or WebSocket), bytes written to
boxes and phones, MCP requests and tool calls by outcome and their time,
web push answers, and the drain's sockets. Every label is a fixed word
(TestCountsNameNobody), and traffic makes no new series and leaves no home,
device, key or address in an hour (TestAnHourOfCountsNamesNobody).

Stacked on #50, whose energy day totals it reads.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(privacy): name the fleet report's health counts

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(ops): a bad key hash leaves the feed off instead of the service down

A malformed FTW_CLOUD_OPS_KEY_SHA256, such as a stray quote or line end
in compose.env, made ops.New fail, and with it the whole service. The
hash is now read by UseKey after New: quotes and white space around it
are dropped, and a hash that is still not 64 hex digits is logged and
leaves /v1/ops answering 404, while boxes and phones are served as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* build: pin srcfl/ftw#1821 at 6a046735

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* build: pin srcfl/ftw#1821 at 5ace4901, its merge with master

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* build: pin srcfl/ftw at 2d7c8eba, #1821 on master

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant