Skip to content

fix(state): durable config writes wait for another writer - #1826

Merged
frahlg merged 2 commits into
masterfrom
fix/durable-config-write-lock-first
Oct 25, 2026
Merged

frahlg merged 2 commits into
masterfrom
fix/durable-config-write-lock-first

Conversation

@frahlg

@frahlg frahlg commented Oct 25, 2026

Copy link
Copy Markdown
Member

Problem

On Fredrik's home box (v0.155.1-beta.1), on 25 October at 06:45:

MyUplink: could not persist rotated refresh_token: database is locked (5) (SQLITE_BUSY)

The same night mpc: persist diagnostic failed and forecast archive: retrying issue hit the same error.

The state database sets busy_timeout(5000), so a writer should wait up to 5 s for another one. SaveDriverSecret did not wait. It runs inside durableConfigWriteContext, which opens a deferred transaction, and it reads the settings row before it writes. SQLite does not wait out the busy timeout for a transaction that already holds a read lock and then asks to write, because waiting there could deadlock. It returns SQLITE_BUSY at once. SaveConfiguration (every settings save) and ReserveVehicleWake read first too.

A rotated OAuth token that is not stored is held only in the driver's memory. After a reload or restart the driver starts from the replaced token, and if the vendor no longer accepts it, the driver cannot sign in until the owner connects again.

Evidence

A test that holds the write lock on another connection for 300 ms:

before after
SaveDriverSecret (reads, then writes) SQLITE_BUSY after 0 s stored after 339 ms
SaveConfig (writes only) waits, stored after 338 ms same

With a 2 s hold, SaveDriverSecret stored after 2.06 s.

Change

durableConfigWriteContext takes the write lock with its first statement, UPDATE config SET value = value WHERE 0, before the caller's function reads anything. That statement changes no row and fires no trigger; the existing trigger test still passes. The existing 5 s busy timeout and the 5 s context then apply to every durable config write.

Tests

  • TestDurableConfigWriteWaitsForAnotherWriter (new): SaveDriverSecret and SaveConfiguration while another connection holds the write lock. Before the fix it fails with the box's exact error.
  • go test ./internal/state: pass.
  • make verify: every package passes except internal/energyreport and internal/statsrelay (the relay report tests). Both fail the same way on master without this change; test(stats): pin the relay client's clock to the fake gateway's #1825 pins those tests' clock.

Pairing

This PR goes with srcfl/device-drivers#193 (myuplink 1.2.4). That PR makes the driver retry a failed store on every poll, so it no longer depends on this one, and this one helps every driver that rotates a token. Merge either first.

🤖 Generated with Claude Code

SaveDriverSecret, SaveConfiguration and ReserveVehicleWake read before
they write inside a deferred transaction. SQLite does not wait out
busy_timeout for a transaction that holds a read lock and then asks to
write; it returns SQLITE_BUSY at once while another connection writes.
On the home box that refused a rotated MyUplink refresh token. Take the
write lock with the first statement, so the 5 s busy timeout applies.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 25, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-25T08:25:30.947032Z 6667923 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@frahlg
frahlg merged commit 0c8eaef into master Oct 25, 2026
22 checks passed
frahlg added a commit to srcfl/device-drivers that referenced this pull request Oct 25, 2026
Once Core waits out a busy database (srcfl/ftw#1826), one failed store can
take 5 s. Stored before the read, that pushed a poll with one timed-out
read to 67.9 s between readings, past the 60 s watchdog. The token is now
stored after the read on every return path, and a rotation during sign-in
is only marked unsaved; the same case gives 57.9 s.

The harness now also checks that, after rotations while the host refuses
the store, the newest token is the one stored.

Signed-off-by: Fredrik Ahlgren <fredrik@sourceful-labs.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
frahlg added a commit to srcfl/device-drivers that referenced this pull request Oct 25, 2026
….4) (#193)

* fix(myuplink): retry a failed read soon and keep a rotated token (1.2.4)

One request that hits the host's 15 s HTTP timeout left 75 s without a
reading (30 s wait, 15 s timeout, 30 s wait), past Core's 60 s watchdog, so
the heat pump was marked offline on every slow request. Retry a failed read
or sign-in once after 2 s; a second failure in a row waits the full 30 s,
so a real outage still shows as offline. A good read asks for 30 s, the
wait Core already applied.

A rotated refresh_token that the host could not store (SQLITE_BUSY) was
held only in memory until the next rotation. Retry the store on every poll
until it succeeds, but not in driver_cleanup, where a reconnect may just
have stored a newer token.

Signed-off-by: Fredrik Ahlgren <fredrik@sourceful-labs.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(myuplink): store a rotated token after the reading, not before

Once Core waits out a busy database (srcfl/ftw#1826), one failed store can
take 5 s. Stored before the read, that pushed a poll with one timed-out
read to 67.9 s between readings, past the 60 s watchdog. The token is now
stored after the read on every return path, and a rotation during sign-in
is only marked unsaved; the same case gives 57.9 s.

The harness now also checks that, after rotations while the host refuses
the store, the newest token is the one stored.

Signed-off-by: Fredrik Ahlgren <fredrik@sourceful-labs.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Signed-off-by: Fredrik Ahlgren <fredrik@sourceful-labs.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant