fix: reliability and performance sweep under load - #51
Conversation
An in-memory SQLite database is destroyed when its last connection closes, and the pool reaped every idle connection after ten minutes.
The idle deadline could only be refreshed by traffic, so a quiet connection was dropped every five minutes even though the client was still there.
Events published while the socket was down were lost for good, leaving the list stale until a manual reload.
Closes #50. Raw source was fetched for every selected message and rendered whole, so a message near the 10 MiB ceiling laid out a 2.4M px document in one frame.
The Headers tab downloaded the whole raw source to read a few kilobytes of headers and re-implemented unfolding in the browser.
`get` selected every column, so a single message view read the full RFC 5322 bytes out of SQLite only for serde to skip them.
A full parse decoded every attachment just to read a handful of authentication headers.
Issued against the pool they reached whichever single connection served them, leaving the rest on SQLite's defaults.
Nothing was sent with a Cache-Control header, so hashed bundles and stored message bytes were re-fetched on every load.
An external-content FTS5 index needs the source row to know which tokens to drop, so deleting it afterwards was a no-op and the index grew forever.
The banner formatted a UTF-16 character count as if it were bytes. The cap itself stays in characters: layout cost tracks glyphs, not encoded size.
Read state, stars and tags change, but the list and single-message responses carried no Cache-Control at all and were open to heuristic caching.
The heartbeat test only ever observed the ping sent on connect, so name it that way and assert the interval-beats-deadline invariant separately.
The cap counted every command, so a client reusing one connection for a bulk send was disconnected after 333 messages. Only unproductive ones count.
A client that fell behind the broadcast channel was served the surviving events and left connected, so its inbox stayed silently wrong. Closing hands the repair to the reconnect path, which already resyncs.
Letting the FTS table drive the join and the ordering lets FTS5 return rowids in descending order natively, so SQLite no longer sorts every match to return one page. Ordering is arrival order now, which ULIDs only approximate to the millisecond, so list and retention follow suit.
The headers and auth endpoints pulled the whole raw blob out of SQLite to parse a header block, so cost scaled with message size instead of header size. A prefix read covers real messages; longer header sections fall back.
The Raw tab rendered 128 KB but downloaded the whole message, so a 1.5 MB mail shipped 1.5 MB to display a twentieth of it. A byte limit keeps the transfer proportional to what is shown.
The JSON export carries read state, stars and tags but had no Cache-Control, while the attachment list is immutable and was not marked cacheable.
The SPA fallback returned index.html with a 200, so an API client could not tell a wrong endpoint from a real response.
A message with attachments writes past SQLite's 1000-page default on its own, so each commit also paid for a checkpoint: two fsyncs and a copy of the WAL, on the connection storing the next message.
Six thousand received messages rendered 90k DOM nodes and held 112 MB, all of it markup: the summaries themselves are 2.9 MB. Windowing renders what is on screen and keeps every row available.
Removing the command cap moved the binding limit on a bulk send to the 300s session timeout, which dropped the socket mid-transaction with no reply. Every await inside a session already carries its own deadline, so the blanket cap only cut off clients that were still delivering.
|
Review of the previously unreviewed commits found one Medium worth fixing here, now done in 573b96c: removing the per-connection command cap had moved the binding limit on a bulk send to the 300s session timeout in `server.rs`, which dropped the socket mid-transaction with no reply. Every await inside a session already carries its own deadline — per-line I/O, the DATA phase, the STARTTLS handshake — so the blanket cap only ever cut off clients that were still delivering. It is gone, with a virtual-time test covering the silent-peer path. Two Low items came along: a malformed `?limit=abc` now fails in the same JSON shape as an out-of-range one instead of falling through to the extractor default, and the websocket test helper reads 64-bit frame lengths. The remaining review item is keyboard navigation into the virtualized list, filed separately as #52 — it is list-navigation work rather than part of this sweep. |
Reliability and performance sweep, driven by load testing an SMTP catcher under the traffic it exists to receive.
Closes #50.
Four things a reviewer cannot infer from the diff:
Message ordering changed from ULID text to arrival order.
list,searchandtrim_to_maxnow order byrowid. ULIDs only sort by time down to the millisecond — within one millisecond the random bits decide, so bursts were shuffled. This is what lets FTS5 return rowids in descending order natively, which is where the search speedup comes from (46k rows, query matching everything: 36 → 2019 req/s, p50 1364 → 23 ms). The exacttotalis preserved; the oldsearch_countjoin intomessageswas a no-op on an external-content index and cost 13 ms.New runtime dependency:
@tanstack/solid-virtual(MIT, one transitive package, no further tail).solid-jswas previously the only runtime dep. Six thousand received messages rendered 90k DOM nodes and held 112 MB — measurement showed only 2.9 MB of that is the message data, the rest is markup. Windowing brings it to 343 nodes and 6 MB with every row still reachable. Bundle cost: +10 KB gzip.The raw endpoint takes
?limit=<bytes>, not aRangeheader.CompressionLayercompresses 206 responses too, which would leaveContent-Rangedescribing the uncompressed bytes while the body is gzip — incorrect per RFC 9110. The alternatives were excludingmessage/rfc822from compression (losing gzip on.emldownloads) or addinghttp-bodyas a direct dependency for a predicate. The query parameter keeps compression and stays correct. Documented indocs/api.yaml.No parallel message processor, deliberately. The single-task insert loop is still the ingest ceiling, but SQLite allows one writer per database: measured 453 insert/s with one writer and 550 with eight, so a worker pool buys ~21% while making WebSocket event order nondeterministic. CPU per 1.37 MB message is down 14.3 → 6.0 ms; throughput is unchanged and disk-bound. The remaining lever is the 1.73× storage amplification —
attachments.contentre-stores decoded what therawblob already holds in base64 — which is a schema change and out of scope here.Also fixed under load: a client reusing one connection was disconnected after 333 messages (the command cap counted every command, not just unproductive ones), and a WebSocket client that fell behind was served the surviving events and left connected, so its inbox stayed silently wrong.
140 tests, clippy clean, every commit builds in isolation.