Collection conformance oracle suite: shared adapter laws for settlement, cancellation, readiness, reset, and ownership #2118

Activity

1. KyleAMathews commented on Oct 28, 2026

   

   KyleAMathews

   on Oct 28, 2026

   CollaboratorAuthor

   More actions

   Folding #2269 into this issue, because they plan the same suite. Three things from #2269 and #2168 extend the plan above.

   Requirements added by the maintainer

   1. Usable by external adapter authors

   Many people write their own sync adapters, so the suite must ship as a public test kit, not only as internal tests.

   • Publish it as a package, for example @tanstack/db-adapter-conformance, or as a @tanstack/db/testing entry point. A third-party adapter author installs it and runs it in their own test runner, with Vitest support at minimum.
   • The public API is the driver interface:
     • capability flags: eager, on-demand, failure kinds, resume, persisted;
     • a factory for the adapter's collectionOptions over a test backend;
     • backend operations: seed, insert, update, delete, read all rows, and inject a failure.
   • Document it: a guide page "Testing your sync adapter" with a minimal example driver, plus JSDoc on every driver hook.
   • The built-in adapters use the same public kit. That proves the kit works and keeps it honest.
   • The suite reports each failing law with its law name, authority, and a shrunk, replayable history (seed and path), in terms an external author can act on.
   • Add reach and coverage reporting per driver, so an author can see which laws their driver declared skipped.

   2. Random data types in the backend, observed on both sides

   • Random schemas and values:
     • The grammar generates the row schema as well as the history: a key of type string, number, or composite where the adapter supports it, and fields drawn from the value domain the adapter declares.
     • Candidate field types: strings including empty and Unicode; integers, floats, negative zero, and large numbers; booleans; null; undefined or a missing field; Date; bigint; nested objects; and arrays.
     • Each driver declares which types its backend round-trips. Electric/Postgres, SQLite, and JSON backends differ.
   • Backend operations and app writes:
     • The history mixes backend-side operations (seed, insert, update, delete, bulk change) with app-side Collection mutations (insert, update, delete through the Collection's handlers) where the adapter supports writes.
   • Two-sided observation: at every quiet checkpoint the oracle reads both sides and checks:
     • Collection to backend: the Collection's visible rows, normalized by the driver's declared serialization rules, equal the backend's rows for the Collection's scope. For on-demand, the scope is the demanded rows.
     • Backend to model: the backend's rows equal the model's expected source after the app's confirmed writes.
     • App writes: a confirmed mutation reaches the backend. A rejected mutation rolls back on both sides, and optimistic state settles to the backend value.
     • Value fidelity: each field's value and type survive the round trip, or differ only by a normalization the driver declares. Examples: Date as an ISO string, bigint as a string, undefined dropped. An undeclared difference is a failure.
   • Interaction with the laws above: reload, failure, on-demand demand changes, and cleanup steps run on the same random data. So "rows equal the source's at reload completion" is checked with real type variety, not only with simple string rows.

   Adapter obligations to enforce (decided in #2168)

   During #2168 the core dropped machinery that tolerated some adapter behaviors, to keep the bundle small. These behaviors are now adapter obligations. The core relies on them and no longer guards them, so the conformance suite must check each one per adapter, with a hostile-backend mutant that breaks it.

   1. An abort ends the load. When the loadSubset signal aborts, the load is over for the core. The core stops listening, so a later success or rejection decides nothing, and no replay waits for it. The adapter must stop its work on abort (the standard AbortSignal contract). Rows it still writes apply as ordinary writes, so it should not write rows for an aborted load.
      Check: a backend that settles after an abort, or keeps writing after it, must not change the request error or the replay's outcome. Its late rows must not reappear after a later replay that excludes them.

   2. A throw is a rejection. A synchronous throw from loadSubset is the same as a rejected promise, a failed load.
      Check: the same history with a throw and with a rejection gives the same observations.

   3. A reload's kept rows are updated, not inserted. During a reload without a snapshot, the Collection keeps the retired run's rows. A new run that sends a row the Collection already holds must write update, not insert. An insert of a held key throws DuplicateKeySyncError.
      Check: a reload of a ready Collection whose new run re-sends every row, with and without changes. No duplicate-key errors, and the rows end equal to the backend's.

   4. Remote deletes during a reload are the adapter's job. The core does not sweep rows after a reload. An adapter that can see deletes must send them during or after the reload: a full snapshot (begin({ snapshot: true }) or truncate()), a stream delete, or ownership, as query-db does. An adapter that cannot see them must document the limit.
      Check: delete a row at the backend while no stream listens, then reload, both eager and on-demand. Either the row is gone at reload completion, or the driver declares the limit and the suite reports the cell as skipped.

   5. Full-state loads are snapshots. A run that sends the full state for the Collection's scope marks that transaction begin({ snapshot: true }) (or uses truncate()). Partial loads, loadSubset results, and metadata-only commits are never marked.
      Check: a mutant driver that marks a partial or resumed commit as a snapshot loses rows. A full-state adapter that does not mark its snapshot keeps rows the backend deleted.

   6. Report the right verb.

      • reportError() while the source still tries, and reportHealthy() or reportHealthy(error) when it recovers.
      • fail() when it stops.
      • Never report a remote failure as a crash. Only a broken local store crashes.
      • Calls after fail() other than markReady() throw, and markReady() after the run is ready or failed is a no-op.

      Check: failure injection per kind gives the documented status and health, with no unhandled rejections.

   7. Stop after cleanup. A retired run's late calls are fenced by the core, but the adapter must release its resources in its cleanup, and its cleanup must not reject for a remote failure. A rejected cleanup of a retired run crashes the Collection.
      Check: a slow or failing remote release during a reload leaves the new run healthy.

   Laws #2168 added that the suite should include

   • A reload of a ready Collection keeps its rows until its new run's snapshot (begin({ snapshot: true }) or truncate()) or its markReady(); the rows then equal the backend's. In feat(db): split Collection status from sync errors; add reload() and cleanup admission #2168 several adapters failed this law before they were ported, and they were only ever tested from error.
   • A local-only reload keeps its rows; localStorage loads each run as a snapshot.
   • Existing receiving witnesses from feat(db): split Collection status from sync errors; add reload() and cleanup admission #2168 become replay examples. They are listed in docs/contributing/oracle-coverage.md under the adapter receivers for reload, the abort receivers, and the Electric, persisted, PowerSync, RxDB, TrailBase, and query-db lanes.
2. KyleAMathews

   added a commit that references this issue on Nov 3, 2026

   Shared eager live-source conformance oracle for sync adapters, and th…

   ...

   9be3c4e