Ordered loader: recover failed ordered requests with a bounded repair instead of a full-source read #2165

Activity

1. KyleAMathews commented on Nov 4, 2026

   

   KyleAMathews

   on Nov 4, 2026

   CollaboratorAuthor

   More actions

   Additional receiving-test evidence on main at 8feab6ea3eaae4456f738da39fa93508ddd5cc01, using the real Query Collection + createCursorPager + live-query setWindow path.

   Measured backend work

   The source uses syncMode: 'on-demand', autoIndex: 'eager', and BasicIndex. Rows have unique ascending numeric ranks; backend pages contain 10 rows. TanStack Query retries are disabled. The failed page throws before returning or writing any rows.

   Each case first loads 10 rows, then requests a 20-row window. Counts below exclude the initial load and, in failure cases, the failed attempt itself. Counts include boundary-equality probes and, for the reload case, reload work.

   Backend size	Path to the 20-row window	Backend calls	Rows returned by backend	Final source size
   100	No failure	2	11	20
   100	Fail expansion, then retry setWindow directly	10	100	100
   100	Fail expansion, await live.reload(), then expand again	3	21	20
   1,000	No failure	2	11	20
   1,000	Fail expansion, then retry setWindow directly	100	1,000	1,000
   1,000	Fail expansion, await live.reload(), then expand again	3	21	20

   All six cases return the correct first 20 rows. After each failed expansion, both the source and visible result still contain only the original 10 rows. Direct retry emits a main-sequence subset request with no limit and the cursor pager exhausts the backend. The successful and reload-then-expand controls do not emit an unlimited main-sequence request. Equality probes are counted separately from that check.

   Reproduction shape

   await live.preload() // orderBy(rank).limit(10)
   failNextPage = true
   await expect(
     live.utils.setWindow({ offset: 0, limit: 20 }),
   ).rejects.toBe(failure)
   
   expect(source.size).toBe(10)
   expect(live.toArray).toHaveLength(10)
   
   // Direct retry: 100 backend calls / 1,000 rows for a 20-row result.
   await live.utils.setWindow({ offset: 0, limit: 20 })
   
   // In a separate fresh fixture, the alternative is bounded:
   // await live.reload() // restores the last settled 10-row window
   // await live.utils.setWindow({ offset: 0, limit: 20 })
   // => 3 calls / 21 returned rows, source.size === 20

   The fixture passes loadSubsetOptions.offset/limit to pager.read(window, context.signal); it serves rank-equality boundary probes exactly. The backend continuation token encodes the next offset. This is an in-memory transport measurement, not a network-latency or database-scan benchmark.

   Interpretation and limits

   This confirms a practical amplification of the documented full-source recovery contract: a single failed continuation can make subsequent direct retry scale with total dataset size rather than the requested window, even when that particular failure wrote nothing.

   It does not establish that bounded repair is safe for generic providers. Partial writes, tied ranks, overlapping invalidations, lease ownership, and automatic repair retries still need the oracle-first work described above. These are fixed receiving examples, not a new recovery law or a production fix.

   The reload control is important: awaited reload followed by expansion stays bounded in this fixture. We encountered this while investigating a real app's reload-based pagination retry, but this test does not reproduce that app's timing-sensitive failure or establish that it has the same cause.

   Validation: six focused runtime tests pass, file ESLint and pnpm lint:changed pass. No production source changes.