Repository navigation
Conversation
Signed-off-by: Segran <goran@andersson-stehag.se>
…ted copies Signed-off-by: Segran <goran@andersson-stehag.se>
Signed-off-by: Segran <goran@andersson-stehag.se>
Signed-off-by: Segran <goran@andersson-stehag.se>
…tail Signed-off-by: Segran <goran@andersson-stehag.se>
miravoss26
left a comment
There was a problem hiding this comment.
Reviewed. Adds audited historical-estimate repair for Home use/Solar gaps plus a "household subtotal" status field for the still-incomplete current day. Fixes #1750.
Correctness: The estimate machinery is careful about what it touches. endpointOriginalValid/planEndpointRepair only fill minutes that are genuinely invalid (missing power, balance mismatch, non-finite/negative values) and bound each gap by real recorded power on both sides within MaxGapMS — anything wider or unbounded gets skipped with a reason, never guessed. ApplyEndpointRepair re-verifies the plan hash against freshly-recomputed evidence before writing (so stale review approvals can't silently apply against changed data), requires an independently-verified backup (checked via os.SameFile, not just path string), detects conflicting overlapping repairs, and is idempotent on re-apply. The CLI tools (ftw-history-endpoints, ftw-history-repair) are local, file-based, human-run with a mandatory --reason — not network-exposed, so no new authz surface.
On the display side, household subtotal is correctly kept out of measured.load_wh and never promoted to the complete total — api_household_subtotal.go's own doc comment states the invariant and the tests (TestHouseholdSubtotalStatusAndDailyKeepIncompleteTotalSeparate) check it round-trips through both /api/status and /api/energy/daily. Web-side changes are plain textContent, no innerHTML/XSS surface.
Test coverage is thorough on both the Go and web sides (estimate-vs-measured separation, malformed-subtotal rejection, quality-label plumbing). No secrets, no new deps, no new network destinations.
Safe to merge from my read.
Missing or rejected source readings and recording gaps can make an entire day's Home use or Solar unknown. This adds explicitly requested, audited historical estimates while preserving the underlying observations and their quality. Fixes #1750; related earlier work: #1667 and #1668.
The API/UI retains a covered Home use subtotal when a complete total remains unknown. Two offline repair methods are available: reconstruction from explicitly selected retained AC source-power series, and an endpoint-mean method for both Home use and Solar. The endpoint helper computes
(power before + power after) / 2 × gap hoursfor incomplete minutes only. It uses valid retained minute source-energy means for Home use and retained ten-second means for Solar, requires both bounds and an explicit maximum endpoint distance, and never assumes zero for missing data. Two observed night-time zeros produce 0 Wh.The endpoint workflow keeps Core collecting throughout backup, preparation and application. After an independent checked SQLite online backup, an operator reviews the hashed plan. Applying it briefly reserves the SQLite writer, rechecks the evidence and atomically publishes the audit metadata plus daily-cache invalidation. No live database is replaced. Original samples, source energy, quality, configuration and newer unrelated measurements remain intact. Repeating a matching plan is idempotent; stale, conflicting or tampered evidence fails closed. Endpoint corrections supersede earlier source-power corrections only in the derived read view; both audits remain available.
Corrections participate in daily totals, system energy-ledger buckets and aggregate charts, remain estimated, and survive portable backup/restore and retention. Estimated Solar is excluded from
measured.pv_wh. Current totals expose completed-through timestamps and do not extrapolate an unfinished writer tail. Raw history and training/control inputs are unchanged. Valid SemVer build metadata retains native update support for the installed current slot; published trial binaries still require an exact release identity.Validation:
make verifypasses (Docker-context check skipped because no Docker daemon), 869 web tests pass, targeted state/API race tests pass. End-to-end state tests cover daytime means, night-time zero endpoints, bounded/unbounded gaps, both flow totals, ledger/chart estimate provenance, live writer-tail bounds, unchanged originals, preservation of later receipts, retry idempotence, atomic rollback, stale/tampered plans, independent backups, retention and portable restore. Existing native update tests cover valid/invalid metadata, incorrect base versions and strict trial identity.This is separate from the forward-looking source-hold and dark-PV policies in #1753/#1755; it does not import or rewrite those policies. Operator workflow:
docs/history-endpoint-repair.mdanddocs/history-repair.md. Independent human UI review remains outstanding; this PR remains a draft.