diff --git a/.changeset/rxdb-ordered-writes.md b/.changeset/rxdb-ordered-writes.md new file mode 100644 index 0000000000..b2bb74ebb3 --- /dev/null +++ b/.changeset/rxdb-ordered-writes.md @@ -0,0 +1,5 @@ +--- +'@tanstack/rxdb-db-collection': patch +--- + +Persist writes to RxDB in the order the application made them. A write now starts only after the earlier write on the same collection fulfills or rejects. Before, a later write could reach RxDB while an earlier one was still pending. A rejected delete now rejects with RxDB's `RxError` for the write error, as a rejected insert does, instead of with an array of write errors. diff --git a/.changeset/rxdb-reject-missing-update.md b/.changeset/rxdb-reject-missing-update.md new file mode 100644 index 0000000000..c9e5138ffb --- /dev/null +++ b/.changeset/rxdb-reject-missing-update.md @@ -0,0 +1,5 @@ +--- +'@tanstack/rxdb-db-collection': patch +--- + +Reject an update whose document no longer exists in RxDB, for example because another tab deleted it while the update waited. The update now fails with `UpdateKeyNotFoundError` and its optimistic change rolls back. Before, the adapter skipped it and reported success, so the write was silently lost. diff --git a/.changeset/rxdb-require-17-6.md b/.changeset/rxdb-require-17-6.md new file mode 100644 index 0000000000..92d702b8c1 --- /dev/null +++ b/.changeset/rxdb-require-17-6.md @@ -0,0 +1,5 @@ +--- +'@tanstack/rxdb-db-collection': patch +--- + +Require RxDB 17.6.0 or later. In RxDB 17.1.0 to 17.5.0, a storage write that throws stops RxDB's incremental write queue (pubkey/rxdb#9025), so the adapter's update never settles and every later update on the collection waits forever. RxDB 17.6.0 fixes this, and the adapter's update now rejects with the storage error. diff --git a/.changeset/trailbase-ordered-confirmed-writes.md b/.changeset/trailbase-ordered-confirmed-writes.md new file mode 100644 index 0000000000..2dbbdee04b --- /dev/null +++ b/.changeset/trailbase-ordered-confirmed-writes.md @@ -0,0 +1,10 @@ +--- +'@tanstack/trailbase-db-collection': patch +--- + +Fix four write bugs. + +- Removing a field in an update (for example `draft.note = undefined`) now sends `null`, so TrailBase clears the column. Before, the update omitted the field, and TrailBase kept the old value. The field's `serialize` function is not called for a removal. +- Writes now reach TrailBase in the order the application made them. A write starts only after TrailBase has answered the earlier write on the same collection. It does not wait for the earlier write's change event. +- A transaction's update and delete requests now go out one at a time, in mutation order. Before, they went out together, so after one was rejected the next write could start while another request was still unanswered. A rejection now ends the transaction, and its later requests are not sent. +- A write is now confirmed only by its own change event. Before, an earlier event for the same record could confirm a later write, so its optimistic value dropped and the earlier value showed until the later event arrived. This change also removes the adapter's periodic cleanup timer. diff --git a/docs/collections/trailbase-collection.md b/docs/collections/trailbase-collection.md index ea1c5ead36..946c3ed190 100644 --- a/docs/collections/trailbase-collection.md +++ b/docs/collections/trailbase-collection.md @@ -52,7 +52,7 @@ The `trailBaseCollectionOptions` function accepts the following options: - `schema`: [Standard Schema](https://standardschema.dev) compatible schema (e.g., Zod, Effect) for client-side validation - `parse`: Object mapping field names to parsing functions that transform data coming from TrailBase -- `serialize`: Object mapping field names to serialization functions that transform data going to TrailBase +- `serialize`: Object mapping field names to serialization functions that transform data going to TrailBase. An update that removes a field (for example `draft.tags = undefined`) sends `null`, which clears the column, without calling that field's function. A `NOT NULL` column rejects such an update. - `onInsert`: Handler function called when items are inserted - `onUpdate`: Handler function called when items are updated - `onDelete`: Handler function called when items are deleted diff --git a/docs/contributing/oracle-coverage.md b/docs/contributing/oracle-coverage.md index fbea9cd98c..811789020c 100644 --- a/docs/contributing/oracle-coverage.md +++ b/docs/contributing/oracle-coverage.md @@ -7448,6 +7448,230 @@ client-cache behavior and is not generated, because the model starts a restarted run empty. A fixed-seed campaign runs the React client (`watchQuery`). +## Adapter conformance: mutations (#2118) + +Owner: `packages/db-adapter-conformance/src/mutation-oracle.ts`, run against +the reference adapter in +`packages/db-adapter-conformance/tests/reference-mutation-oracle.property.test.ts` +(`adapter-conformance.mutations.reference`, with a structured-clone, a JSON, +and a SQL provider), against RxDB in +`packages/rxdb-db-collection/tests/mutation-conformance-oracle.property.test.ts` +(`adapter-conformance.mutations.rxdb`), against TrailBase in +`packages/trailbase-db-collection/tests/mutation-conformance-oracle.property.test.ts` +(`adapter-conformance.mutations.trailbase`), against PowerSync in +`packages/powersync-db-collection/tests/mutation-conformance-oracle.property.test.ts` +(`adapter-conformance.mutations.powersync`), against Electric in +`packages/electric-db-collection/tests/mutation-conformance-oracle.property.test.ts` +(`adapter-conformance.mutations.electric`), and against Convex in +`packages/convex-db-collection/tests/mutation-conformance-oracle.property.test.ts` +(`adapter-conformance.mutations.convex`). It checks auto-transaction +writes on a ready Collection whose adapter persists them itself: MUT-1 (the +public rows show the optimistic state while the mutation function runs, and +no publication shows the earlier synced value when it fulfills), MUT-2 (a +provider rejection rejects `when('settled')` with the provider's error and +publishes exactly the inverse), MUT-8, MUT-10, and MUT-13 (each write is one +provider request of its type with the authored row, in author order, issued +only after the earlier write is decided), and FID-5 (at a quiescent cut the +public rows, `collection.base`, and the provider's rows are identical and +equal the fold of accepted writes). The provider answers each request by a +plan the check hands it before the write: accept or reject, at once or held +until a `release`. Publications compare row values; a publication of the same +row values, such as core's `$hasPendingWrites` update when a write is +confirmed, counts as no change here (the virtual-row-field owner judges it). +The model follows the glossary's accepted sync transaction: an accepted +write's provider copy is held while any mutation is persisting, and the +write's held optimistic row stays until it applies. A driver declares how its +provider stores an authored `undefined` field (`undefinedField`, from FID-1's +value domain): `own`, as a structured clone keeps it, or `absent`, as a JSON +document must. Under `absent`, a removed field is absent from the request and +the provider's copy, and the copy publishes one more change of row values when +it replaces the held optimistic row. With `server-rewrites`, an accepted +insert's or update's plan can also rewrite the stored row (normalize the +title, drop `note`, or set `note` as a server default): the request still +carries the authored row (MUT-8), and the provider's copy, not the authored +row, is the row's value once it applies (MUT-1, FID-5). With `field-updates` +(the law set's capability), an update sends only its changed fields and the +provider merges them into its row as it is when the write is accepted; a +removed field is sent in the provider's representation, `null` for a SQL +provider (`undefinedField: 'null'`, which also sends a complete row with +every column). With `held-echoes`, the provider can answer an accepted write +before its change feed delivers the copy, at a `deliver` step: under MUT-7 the +mutation stays pending until its copy arrives. MUT-10's "the earlier one +decides" is the provider's answer, so a later write's request is owed once +the earlier write is answered, without waiting for its copy (maintainer +decision 2026-11-03: core does not chain independent transactions; an +application that needs that chains them itself). With `application-handlers` +the application's own handlers persist writes and the adapter only confirms +them, so each write's request goes out when it is made and MUT-10 is the +application's; with `ordered-answers` the provider answers requests in arrival +order. With `log-ordered-echoes` the change feed delivers copies in commit +order, so a withheld copy holds back every later one. When one step settles +several writes, they settle in an order no law fixes, so the publication check +compares each row's own sequence of values instead of the whole publication +sequence (by the same maintainer decision): it still rejects an earlier synced +value or an extra change on any row. A step that settles one write keeps the +whole-sequence comparison. A driver can declare several `rejectionKinds`, each reached. Reach: each of insert, update, and delete +accepted, a rejected write, a held write, a write queued behind a held one, a +held write rejected, a commit by another client, an accepted field removal, a +held optimistic row, a copy the provider rewrote, an answered write whose copy +is withheld, each declared rejection kind, and a quiescent cut after writes, +at least five histories each per campaign; a grammar arm generates +updates without `note`, so removals are common. Calibration: twelve reference +mutants each fail +with their law (an applied copy after the mutation function settles, a +swallowed or wrapped rejection, a duplicated insert request, writes that do +not wait for the earlier one, a skipped delete, an update that sends only +changed fields, under a JSON provider a change feed that carries the +authored row instead of the stored copy, and an adapter that writes the +authored row into its synced rows after the provider accepts, and under a SQL +provider an update that sends the complete row, one that drops a removed +field, and a mutation function that settles when the provider answers instead +of when its copy arrives). The write-back mutant survives the fixed campaign +without `server-rewrites` and fails FID-5 with it. Before the held-row rule, the +JSON-provider campaign failed MUT-1 on a fulfilled removal behind a pending +write. + +RxDB (`undefinedField: 'absent'`, `server-rewrites`, `merged-updates`): an +update writes the fields its authored row names and keeps the others, as +`incrementalPatch` does (maintainer decision 2026-11-03, revising the law +set's complete-row MUT-8 clause for RxDB); a grammar arm stores a field the +client has not seen, from a held write's rewrite, before an update that does +not name it (reach `merge-kept-field`, 7 to 13 of 300 histories per seed), +and an RxDB that replaces the row through `incrementalModify` fails MUT-8. +Before the mode, seed 371775937 of the random campaign failed MUT-8 there. +The driver runs a real +RxDB database on memory storage, and a rewrite is RxDB's `preInsert` and +`preSave` hooks; the request keeps the document as the adapter sent it. With +the hooks made no-ops, the fixed campaign fails FID-5. The provider wraps that storage's `bulkWrite`, stores +documents as JSON as RxDB's persistent storages do, rejects a request with a +per-row write error (status 422, as schema validation reports), and reads its +rows from the storage instance rather than RxDB's document cache. MUT-2's +relation is the `RxError` whose `parameters.writeError` is the storage's error +object. On the parent of the fix, 38 of the fixed campaign's 300 histories +failed MUT-10 (a write reached RxDB while an earlier one was pending) and 24 +failed MUT-2 (a rejected delete rejected with an array of write errors). The +adapter now runs each write after the earlier one is decided and rejects a +failed delete with RxDB's `RxError`. RxDB rejection kinds (`rejectionKinds`): a per-row write error and a +`bulkWrite` that throws, as a failing storage does; each must be reached. On +RxDB 17.1.0 to 17.5.0 a thrown storage error wedges the incremental write +queue: the throw is an unhandled rejection, `incrementalPatch` never settles, +and every later incremental write on the collection waits forever +(pubkey/rxdb#9025, fixed in RxDB 17.6.0 by pubkey/rxdb#9058). On 17.1.0 both +campaigns failed HEALTH-5 at an update whose storage write throws. The adapter +keeps `incrementalPatch` and its conflict retry, and its RxDB dev dependency +and peer range are now 17.6.0 and `>=17.6.0` (maintainer decision +2026-11-03); both campaigns pass. A conflict (status 409) is not generated. An update of a +document that another client deleted while the update was pending now +rejects with `UpdateKeyNotFoundError` (maintainer decision 2026-11-03; it +used to fulfill without persisting); commits by another client while a write +is pending are outside the grammar (MUT-11), so `rxdb.test.ts` holds the +supplemental witness, RED on the parent. +A removed field is an own `undefined` in the authored row (maintainer +decision 2026-11-03, keeping the documented rule in +`packages/db/tests/proxy-detachment-contract-oracle.test.ts`); the core owner +`optimistic-history-oracle` now generates that removal (an optional `d`, set +and then removed with `delete draft.d`), compares rows by own fields, and +counts its reach (`removedFieldEdits`). A write may follow pending ones on +its key (reach `same-key-pending`, and a grammar arm that writes one key twice +in a row) when their plans accept them; for an adapter that does not order +writes, only once they are answered. After a rejected insert or delete, what a +provider does with the next write on that key is the law set's per-adapter +`unknownKeyUpdate` (MUT-10), which no driver declares, so those histories are +not generated. Every provider keeps one key's copies in commit order, so a +copy waits behind an undelivered one for its key. An update removes `note` +only when the held row holds a note value, in the model and every writer. +Limits: commits by another client only while no mutation is pending, so MUT-11 +is not checked (a commit held behind a pending mutation is core's MUT-5); no +manual or multi-mutation transactions, so MUT-8's mutation order within one +transaction is not checked; no schema validation or update that changes no +field; a newer backend value in a disjoint field; server-generated keys. + +TrailBase (`undefinedField: 'null'`, `field-updates`, `held-echoes`, and every +other mutation capability): the driver's RecordApi acts as a TrailBase +service with SQL rows, records each `createBulk` record, `update`, and +`delete` as JSON, as the SDK sends it, and sends change events at once or at +`deliver`. The application writes an absent `note` as `null` and removes a +held one by assigning `undefined`; removal by assigning `null` is not +generated. On the parent of the fix, 37 of the fixed campaign's 300 histories +failed MUT-8 (a removed field was sent as an empty patch, so the service kept +it), 27 failed MUT-10 (writes did not wait for the earlier one), and 5 failed +MUT-1 (a write was confirmed by an earlier event on the same key, so its +optimistic row dropped before its own event arrived). The adapter now sends +a removed field as `null` and starts each write after the earlier one's HTTP +response, not after its change event; the version that waited for the event +fails the fixed campaign (a request owed after the answer is missing). Its +change stream is in commit order (`log-ordered-echoes`). The adapter confirms +the n-th answered write on a key by the n-th change event on that key, so an +earlier write's late event cannot confirm a later write on the key; with same-key +histories, the version that confirmed by any event after the write began +failed MUT-1 and MUT-7, and a mutant keeping it is rejected. Events from +another client while a write is pending are outside these histories (MUT-11). +A transaction's update and delete requests go out one at a time in mutation +order, so the next write starts only after each is answered, and a rejection +ends the transaction there; `trailbase.test.ts` holds a supplemental witness +(RED on the parent: the next write started before a sibling request was +answered). Multi-mutation transactions are not generated, so this and the +law for a rejection partway through a transaction (whether later requests +still go out, and what partial acceptance leaves) remain open: follow-up for +this owner, decision 2026-11-03. +The record API is a test double that bypasses the SDK's fetch and SSE +decoding. + +PowerSync (`undefinedField: 'null'`, `field-updates`, `write-rejection`, +`foreign-commits`): the driver runs a real local PowerSync database, which is +the adapter's provider; uploading to a PowerSync service is the SDK's upload +queue and is outside this owner. A request is one `INSERT`, `UPDATE`, or +`DELETE` the adapter executes on the table inside its write transaction; a +rejection throws from that statement and rolls the transaction back. The +database's write lock orders the adapter's writes, which meets MUT-10 under +the decided-on-answer rule, and each write settles once the adapter's diff +trigger has observed it (MUT-7); the watcher's 30 ms throttle makes that +copy arrive late, and a mutant that settles without waiting for it fails +MUT-1. The adapter needed no change. The propagation step waits until every +write's receipt settles and the watcher has drained. Limits: `held-writes` is +not declared, because a held statement holds the one write lock that the +adapter's diff flush also needs, so it would delay earlier writes' +confirmation, a coupling the model does not represent; `held-echoes` is not +declared, because one watcher callback applies every committed write; +`server-rewrites` is not declared, because the local database rewrites +nothing; removal by assigning `null` and multi-mutation transactions are not +generated. + +Electric (`application-handlers`, `log-ordered-echoes`, `held-echoes`, +`undefinedField: 'null'`, `write-rejection`, `held-writes`, `foreign-commits`, +`server-rewrites`): Electric does not persist writes. The driver's application +handlers write the complete SQL row through a backend API that answers by plan +with the commit's txid, then await the adapter's `awaitTxId`; the shape stream +is the real `@electric-sql/client` `ShapeStream` over a controlled +`fetchClient`, each commit one batch with its txid, in commit order, stalled +at a held echo until `deliver`. This checks what Electric owns, confirmation +through its shape stream (MUT-7), with the handlers' requests and the +backend's rows as the application's. GREEN with no adapter change; a mutant +whose txid evidence check accepts any txid fails MUT-1 at the first insert. +The existing `tests/documented-mutations-oracle.test.ts` owner keeps the +documented guidance handlers. Limits: one sync run (no reconnect or +must-refetch while a write waits); `awaitMatch` is not exercised. + +Convex (`application-handlers`, `ordered-answers`, `log-ordered-echoes`, +`held-echoes`, `undefinedField: 'absent'`, `write-rejection`, `held-writes`, +`foreign-commits`, `server-rewrites`): the driver runs the real Convex +JavaScript client over a controlled WebSocket and plays the server. The +application's handlers call one Convex mutation per write with the complete +row as Convex values; the server runs one client's mutations in arrival order, +answers each by plan (a rejection is a failed `MutationResponse`), and commits +an acceptance at a new timestamp, whose `Transition` carries the query's +complete result after the response. The client resolves a mutation only at a +transition that reaches its timestamp, so transitions are the copies; a held +echo stalls them. GREEN with no adapter change; an adapter that does not +await the application's handler fails MUT-1. Limits: an adapter that skips +its wait for the source after the handler survives both campaigns, because +the histories keep one subscription whose result the client applies before it +resolves the mutation; the cached-listener path that wait protects is owned by +`tests/collection.test.ts` and the ownership oracle, which reject that mutant. +The JavaScript client only; no reconnect while a mutation waits. +||||||| 9be3c4ed2 + + ### Ordered-loader authority CI counterexample, 2026-11-03 Owner: `packages/db/tests/query/ordered-source-loader-state-oracle.test.ts`, diff --git a/packages/convex-db-collection/tests/mutation-conformance-driver.ts b/packages/convex-db-collection/tests/mutation-conformance-driver.ts new file mode 100644 index 0000000000..0c26366ba2 --- /dev/null +++ b/packages/convex-db-collection/tests/mutation-conformance-driver.ts @@ -0,0 +1,430 @@ +/** + * Production driver for the shared mutation oracle + * (`packages/db-adapter-conformance/src/mutation-oracle.ts`). It runs the real + * Convex adapter in eager mode over the real Convex JavaScript client, whose + * WebSocket the driver controls, as `conformance-driver.ts` does: the + * client's query set, mutation queue, timestamps, and subscriptions are real, + * and the driver plays the server. + * + * Convex does not persist writes: the application's handlers call a Convex + * mutation, and the adapter wraps each handler so it settles only once its + * subscription holds the result after that mutation. The driver supplies + * such an application: + * + * - the provider is the server's table. A write request is one `Mutation` + * message; its args carry the complete row as Convex values, so a field + * whose value is `undefined` is absent (`undefinedField: 'absent'`); + * - the server runs one client's mutations in arrival order + * (`ordered-answers`) and answers each by plan. A rejection is a failed + * `MutationResponse` carrying the plan's message; an acceptance commits at + * a new timestamp and answers with it; + * - each commit reaches the query as one `Transition` to its timestamp, + * carrying the complete result. The client resolves a mutation only once a + * transition reaches its timestamp, so transitions are the copies, in + * commit order (`log-ordered-echoes`); a held echo stalls them at that + * commit until `deliver` (`held-echoes`); + * - a rewrite (`server-rewrites`) is the mutation storing a changed row; a + * commit by another client is a server commit that is not the client's; + * - the adapter cannot order the application's handlers + * (`application-handlers`), so each write's request goes out when it is + * made. + * + * Rows are Convex values: an absent `note` is a missing field. Limits: the + * JavaScript client only; no reconnect while a mutation waits. + */ +import { createCollection } from '@tanstack/db' +import { ConvexClient } from 'convex/browser' +import { makeFunctionReference } from 'convex/server' +import { convexToJson, jsonToConvex } from 'convex/values' +import { convexCollectionOptions } from '../src/index' +import { + acceptAtOnce, + observedView, + rewriteRow, + settleWhenQuiet, +} from '../../db-adapter-conformance/src' +import type { JSONValue, Value } from 'convex/values' +import type { + ConformanceRow, + MutationDriver, + MutationFixture, + MutationProvider, + ProviderChange, + SettleCut, + WritePlan, + WriteRequest, +} from '../../db-adapter-conformance/src' + +const rowsQuery = makeFunctionReference< + `query`, + Record, + Array +>(`rows:list`) + +const writeMutation = makeFunctionReference< + `mutation`, + { request: WriteRequest }, + null +>(`rows:write`) + +function timestamp(value: number): string { + const bytes = Buffer.alloc(8) + bytes.writeBigUInt64LE(BigInt(value)) + return bytes.toString(`base64`) +} + +/** The WebSocket the Convex client opens; the driver plays the server. */ +class Socket { + static provider: ConvexMutationProvider | undefined + onopen: (() => void) | null = null + onmessage: ((event: { data: string }) => void) | null = null + onclose: ((event: { code: number; reason: string }) => void) | null = null + onerror: (() => void) | null = null + readonly sent: Array> = [] + private readonly provider: ConvexMutationProvider + constructor(_url: string) { + this.provider = Socket.provider! + this.provider.socket = this + queueMicrotask(() => this.onopen?.()) + } + send(data: string) { + const message = JSON.parse(data) as Record + this.sent.push(message) + // A server answers after the message leaves; the client registers a + // mutation as in flight only after it sends it. + queueMicrotask(() => this.provider.receive(message)) + } + close() { + queueMicrotask(() => this.onclose?.({ code: 1000, reason: `` })) + } + deliver(message: unknown) { + this.onmessage?.({ data: JSON.stringify(message) }) + } +} + +type Answer = { run: () => void; held: boolean } + +class ConvexMutationProvider implements MutationProvider { + readonly client: ConvexClient + socket: Socket | undefined + private readonly table = new Map() + private readonly plans: Array = [] + private readonly held: Array = [] + /** Requests not yet answered, in arrival order. */ + private readonly unanswered: Array = [] + private readonly log: Array = [] + /** Commits whose transitions are stalled; `own` marks a held echo. */ + private readonly stalled: Array<{ + rows: Array + ts: number + own: boolean + }> = [] + private time = 0 + /** The timestamp of the last transition sent. */ + private sentTs = 0 + /** The query-set version the client last saw, and the one it asked for. */ + private querySet = 0 + private requestedQuerySet = 0 + private queryId: number | undefined + private activity = 0 + + constructor(seed: ReadonlyArray) { + for (const row of seed) this.table.set(row.id, row) + Socket.provider = this + this.client = new ConvexClient(`https://mutations.convex.cloud`, { + webSocketConstructor: Socket as unknown as typeof WebSocket, + unsavedChangesWarning: false, + logger: false, + }) + } + + private result(): Array { + return [...this.table.values()].sort((a, b) => a.id - b.id) + } + + /** A transition from the last one sent to `ts`, with the query's result. */ + private transition( + ts: number, + rows?: Array, + extra: Array = [], + ): void { + this.socket!.deliver({ + type: `Transition`, + startVersion: { + querySet: this.querySet, + ts: timestamp(this.sentTs), + identity: 0, + }, + endVersion: { + querySet: this.requestedQuerySet, + ts: timestamp(ts), + identity: 0, + }, + modifications: [ + ...extra, + ...(rows && this.queryId !== undefined + ? [ + { + type: `QueryUpdated`, + queryId: this.queryId, + value: convexToJson(rows as unknown as Value), + logLines: [], + journal: null, + }, + ] + : []), + ], + }) + this.sentTs = ts + this.querySet = this.requestedQuerySet + this.activity++ + } + + /** A message from the client. */ + receive(message: Record): void { + if (message.type === `ModifyQuerySet`) { + this.requestedQuerySet = message.newVersion as number + const modifications = message.modifications as Array<{ + type: string + queryId: number + }> + for (const modification of modifications) { + if (modification.type === `Add`) this.queryId = modification.queryId + } + const added = modifications.some((m) => m.type === `Add`) + const removed = modifications + .filter((m) => m.type === `Remove`) + .map(({ queryId }) => ({ type: `QueryRemoved`, queryId })) + // The server acknowledges a query-set change in a new transition. + queueMicrotask(() => + this.transition( + ++this.time, + added && this.stalled.length === 0 ? this.result() : undefined, + removed, + ), + ) + return + } + if (message.type !== `Mutation`) return + const requestId = message.requestId as number + const { request } = jsonToConvex( + (message.args as Array)[0]!, + ) as unknown as { request: WriteRequest } + this.log.push(request) + this.activity++ + const plan = this.plans.shift() ?? acceptAtOnce + const answer: Answer = { + held: plan.held, + run: () => { + if (plan.outcome === `reject`) { + this.socket!.deliver({ + type: `MutationResponse`, + requestId, + success: false, + result: plan.error.message, + logLines: [], + }) + return + } + // A server answers a mutation before the transition that carries + // it; the client resolves the mutation at that later transition. + const { ts, publish } = this.commit( + request.type === `delete` + ? { type: `delete`, id: request.id } + : { + type: `upsert`, + row: rewriteRow(request.row as ConformanceRow, plan.rewrite), + }, + plan.echoHeld, + ) + this.socket!.deliver({ + type: `MutationResponse`, + requestId, + success: true, + result: null, + ts: timestamp(ts), + logLines: [], + }) + publish() + }, + } + this.unanswered.push(answer) + if (plan.held) this.held.push(answer) + else if (this.unanswered[0] === answer) this.answer(answer) + } + + /** Answers a request, then each one behind it that is not held. */ + private answer(answer: Answer): void { + this.unanswered.splice(this.unanswered.indexOf(answer), 1) + answer.run() + this.activity++ + while (this.unanswered[0] && !this.unanswered[0].held) + this.answer(this.unanswered[0]) + } + + /** + * Commits at a new timestamp. `publish` sends its transition, or stalls + * it behind a held echo. + */ + private commit( + change: ProviderChange, + holdEcho: boolean, + ): { ts: number; publish: () => void } { + if (change.type === `delete`) this.table.delete(change.id) + else this.table.set(change.row.id, change.row) + const ts = ++this.time + const rows = this.result() + return { + ts, + publish: () => { + if (holdEcho || this.stalled.length > 0) + this.stalled.push({ rows, ts, own: holdEcho }) + else this.transition(ts, rows) + }, + } + } + + // The oracle's side. + + plan(plan: WritePlan): void { + this.plans.push(plan) + } + + release(): void { + const answer = this.held.shift() + if (answer) this.answer(answer) + } + + /** Transitions resume at the oldest held echo, up to the next one. */ + deliver(): void { + const first = this.stalled.shift() + if (!first) return + this.transition(first.ts, first.rows) + while (this.stalled[0] && !this.stalled[0].own) { + const next = this.stalled.shift()! + this.transition(next.ts, next.rows) + } + } + + /** Answers every held request and resumes the transitions. */ + drain(): void { + while (this.held.length > 0) this.release() + while (this.stalled.length > 0) this.deliver() + } + + foreignCommit(change: ProviderChange): void { + this.commit(change, false).publish() + } + + rows(): ReadonlyArray { + return this.result() + } + + requests(): ReadonlyArray { + return this.log + } + + /** + * The propagation step: every write's receipt settled; while a request or + * an echo is held, until the requests, transitions, and receipts are + * quiet. The client applies a transition within a few turns. + */ + settle = ({ unsettled }: SettleCut) => + settleWhenQuiet({ + done: () => unsettled() === 0, + blocked: () => this.held.length > 0 || this.stalled.length > 0, + activity: () => `${this.activity} ${unsettled()}`, + }) + + dispose = () => this.client.close() +} + +let sequence = 0 + +export function convexMutationDriver(): MutationDriver { + return { + adapter: `@tanstack/convex-db-collection (eager, javascript client)`, + capabilities: new Set([ + `write-rejection`, + `held-writes`, + `foreign-commits`, + `server-rewrites`, + `held-echoes`, + `log-ordered-echoes`, + `application-handlers`, + `ordered-answers`, + ]), + undefinedField: `absent`, + // The client creates a failed mutation's error from the server's + // message, which ends with the injected failure's index. + matchesFailure: (reported, failure) => + reported instanceof Error && + new RegExp(`${failure.message}(?!\\d)`).test(reported.message), + create(seed): MutationFixture { + const provider = new ConvexMutationProvider(seed) + // The application's handlers: one Convex mutation per write. + const persist = async (requests: Array) => { + for (const request of requests) + await provider.client.mutation(writeMutation, { request }) + } + const collection = createCollection( + convexCollectionOptions({ + id: `convex-mutations-${++sequence}`, + client: provider.client, + query: rowsQuery, + queryArgs: {}, + getKey: (row) => row.id, + onInsert: ({ transaction }) => + persist( + transaction.mutations.map((mutation) => ({ + type: `insert`, + id: mutation.modified.id, + row: mutation.modified, + })), + ), + onUpdate: ({ transaction }) => + persist( + transaction.mutations.map((mutation) => ({ + type: `update`, + id: mutation.modified.id, + // The complete row; Convex values drop an `undefined` field. + row: mutation.modified, + })), + ), + onDelete: ({ transaction }) => + persist( + transaction.mutations.map((mutation) => ({ + type: `delete`, + id: mutation.original.id, + })), + ), + }), + ) + return { + collection: observedView(collection, { + row: (row) => row, + key: (key) => key as number, + }), + base: () => collection.base.values(), + writer: { + insert: (row) => collection.insert(row), + update: (row) => + collection.update(row.id, (draft) => { + draft.title = row.title + if (Object.hasOwn(row, `note`)) draft.note = row.note + // Only a held note value is removed. + else if (typeof draft.note === `string`) delete draft.note + }), + delete: (id) => collection.delete(id), + }, + provider, + settle: provider.settle, + drain: () => provider.drain(), + dispose: async () => { + await collection.cleanup() + await provider.dispose() + }, + } + }, + } +} diff --git a/packages/convex-db-collection/tests/mutation-conformance-oracle.property.test.ts b/packages/convex-db-collection/tests/mutation-conformance-oracle.property.test.ts new file mode 100644 index 0000000000..482f64bae2 --- /dev/null +++ b/packages/convex-db-collection/tests/mutation-conformance-oracle.property.test.ts @@ -0,0 +1,76 @@ +import { fc } from '@fast-check/vitest' +import { describe, it } from 'vitest' +import { + oraclePropertyOptions, + oracleReplayTest, + oracleRuns, +} from '../../db/tests/oracle-config' +import { declareOracle } from '../../db/tests/oracle-declaration' +import { createOracleReach } from '../../db/tests/oracle-reach' +import { + checkMutationHistory, + mutationHistoryArb, + mutationModel, + mutationProperty, + mutationReachStatesFor, +} from '../../db-adapter-conformance/src' +import { convexMutationDriver } from './mutation-conformance-driver' + +/** + * # Convex on the shared mutation oracle + * + * A receiving driver for `packages/db-adapter-conformance`. The shared + * module states the laws (MUT-1, MUT-2, MUT-7, MUT-8, MUT-10, MUT-13, FID-5, + * HEALTH-5), the model, the history grammar, and the check. This file + * supplies only the Convex production driver in + * `mutation-conformance-driver.ts` and the campaigns; it states no expected + * results. + * + * Convex does not persist writes: the driver's application handlers call a + * Convex mutation, and the adapter's wrapped handler settles only once its + * subscription holds the result after that mutation (MUT-7). The server runs + * one client's mutations in order (`ordered-answers`), can reject or hold + * one, rewrite a stored row, and stall its transitions at a commit + * (`held-echoes`, `log-ordered-echoes`); another client can write; and the + * adapter does not order the application's handlers (`application-handlers`). + */ + +const property = `adapter-conformance.mutations.convex` +const driver = convexMutationDriver() +const historyArb = mutationHistoryArb(driver) +const runs = 300 +const campaignTimeoutMs = 120_000 + +async function campaign(options: fc.Parameters<[unknown]>) { + const reach = createOracleReach(property, mutationReachStatesFor(driver), { + minimum: 5, + }) + await fc.assert(mutationProperty(driver, reach), options) + return reach +} + +describe(`mutation conformance: Convex`, () => { + oracleReplayTest(it)( + `conforms on a fixed-seed campaign`, + async () => { + const reach = await campaign({ seed: 2118, numRuns: oracleRuns(runs) }) + reach.assertReached() + }, + campaignTimeoutMs, + ) + + oracleReplayTest(it, property)( + `conforms on a random campaign or a replayed seed`, + async () => { + const reach = await campaign(oraclePropertyOptions(runs, property)) + reach.assertReached() + }, + campaignTimeoutMs, + ) +}) + +declareOracle({ + check: [checkMutationHistory, mutationProperty], + model: [mutationModel.step, mutationModel.visible], + grammar: [historyArb], +}) diff --git a/packages/db-adapter-conformance/MIGRATION.md b/packages/db-adapter-conformance/MIGRATION.md index 403ab601d7..f0327710b3 100644 --- a/packages/db-adapter-conformance/MIGRATION.md +++ b/packages/db-adapter-conformance/MIGRATION.md @@ -17,7 +17,7 @@ reviewed change. | Family | Status | Laws | | ------------------------------------- | ----------------------------------------- | -------------------------------------------------------------------------------------------------------------------- | | Eager live source | Built (`src/eager-live-source-oracle.ts`) | READY-2/3/4/8/10, HEALTH-1/2/5, PUB-1/2/3/5/7/9, FID-1/4, LIFE-2/5/10, EVID-1/2/3, RETRY-1 (eager), RECON-1, RENEW-1 | -| Mutations and confirmation | Needed | MUT-1/2/4/5/7/10/13, FID-5 | +| Mutations and confirmation | Built (`src/mutation-oracle.ts`) | MUT-1/2/4/5/7/10/13, FID-5 | | On-demand loading | Needed | DEM-1 to DEM-23, MAP-1 to MAP-4 | | Reload and retry | Needed | RELOAD-1 to RELOAD-5; RETRY-1 and HEALTH-2 beyond the eager source's retried failures | | Client cache and refetch | Needed | CACHE-1 to CACHE-8, REF-1 to REF-7, DIR-1 to DIR-5 | @@ -27,28 +27,65 @@ reviewed change. ## Adapters -| Adapter | Families that apply (section A) | Driver status | -| ----------------- | ------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| Neon Realtime | eager live source, mutations | Eager: RED (READY-4, HEALTH-1, and LIFE-10 behind them); adapter update pending | -| Electric | eager live source, mutations, on-demand, source shapes, reload and retry, persistence | Eager: GREEN (2026-10-30) after three fixes, per-batch commits, typed Postgres columns, over the installed client. Driver `electric-db-collection/tests/conformance-driver.ts`, property `adapter-conformance.eager-live-source.electric`. Other families: not started | -| PowerSync | eager live source, mutations, on-demand | Eager: GREEN (2026-10-30) after a fix, per-batch commits, over real local SQLite. Driver `powersync-db-collection/tests/conformance-driver.ts`, property `adapter-conformance.eager-live-source.powersync`. Mutations, on-demand: not started | -| RxDB | eager live source, mutations | Eager: GREEN (2026-10-29) after a fix, per-change commits, `listen-before-fetch`. Driver `rxdb-db-collection/tests/conformance-driver.ts`, property `adapter-conformance.eager-live-source.rxdb`. Mutations: not started | -| TrailBase | eager live source, mutations, on-demand | Eager: GREEN (2026-10-29), per-change commits. Driver `trailbase-db-collection/tests/conformance-driver.ts`, property `adapter-conformance.eager-live-source.trailbase`; declares `duplicate-key-delivery` and `source-failure-after-ready`. Mutations, on-demand: not started | -| Convex | eager live source, mutations, on-demand, source shapes | Eager: GREEN (2026-11-02) after a fix, per-batch commits, complete results, over the real Convex JavaScript client and a controlled WebSocket. Driver `convex-db-collection/tests/conformance-driver.ts`, property `adapter-conformance.eager-live-source.convex`. Mutations, on-demand, source shapes: not started | -| query-db | client cache and refetch, on-demand, mutations, reload and retry, persistence | Not started; not a push source, so the eager family does not apply | -| local-only | local and shared stores, mutations | Not started | -| localStorage | local and shared stores, mutations | Not started | -| Persisted wrapper | persistence wrapper | Not started | -| Storage hosts (8) | storage hosts | Not started | +| Adapter | Families that apply (section A) | Driver status | +| ----------------- | ------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Neon Realtime | eager live source, mutations | Eager: RED (READY-4, HEALTH-1, and LIFE-10 behind them); adapter update pending | +| Electric | eager live source, mutations, on-demand, source shapes, reload and retry, persistence | Eager: GREEN (2026-10-30) after three fixes, per-batch commits, typed Postgres columns, over the installed client. Driver `electric-db-collection/tests/conformance-driver.ts`, property `adapter-conformance.eager-live-source.electric`. Mutations: GREEN (2026-11-03), no adapter change, `application-handlers`, `log-ordered-echoes`, `held-echoes`. Driver `electric-db-collection/tests/mutation-conformance-driver.ts`, property `adapter-conformance.mutations.electric`. Other families: not started | +| PowerSync | eager live source, mutations, on-demand | Eager: GREEN (2026-10-30) after a fix, per-batch commits, over real local SQLite. Driver `powersync-db-collection/tests/conformance-driver.ts`, property `adapter-conformance.eager-live-source.powersync`. Mutations: GREEN (2026-11-03), no adapter change, `undefinedField: 'null'`, `field-updates`. Driver `powersync-db-collection/tests/mutation-conformance-driver.ts`, property `adapter-conformance.mutations.powersync`. On-demand: not started | +| RxDB | eager live source, mutations | Eager: GREEN (2026-10-29) after a fix, per-change commits, `listen-before-fetch`. Driver `rxdb-db-collection/tests/conformance-driver.ts`, property `adapter-conformance.eager-live-source.rxdb`. Mutations: GREEN (2026-11-03) after a fix, `undefinedField: 'absent'`. Driver `rxdb-db-collection/tests/mutation-conformance-driver.ts`, property `adapter-conformance.mutations.rxdb` | +| TrailBase | eager live source, mutations, on-demand | Eager: GREEN (2026-10-29), per-change commits. Driver `trailbase-db-collection/tests/conformance-driver.ts`, property `adapter-conformance.eager-live-source.trailbase`; declares `duplicate-key-delivery` and `source-failure-after-ready`. Mutations: GREEN (2026-11-03) after a fix, `undefinedField: 'null'`, `field-updates`, `held-echoes`. Driver `trailbase-db-collection/tests/mutation-conformance-driver.ts`, property `adapter-conformance.mutations.trailbase`. On-demand: not started | +| Convex | eager live source, mutations, on-demand, source shapes | Eager: GREEN (2026-11-02) after a fix, per-batch commits, complete results, over the real Convex JavaScript client and a controlled WebSocket. Driver `convex-db-collection/tests/conformance-driver.ts`, property `adapter-conformance.eager-live-source.convex`. Mutations: GREEN (2026-11-03), no adapter change, `application-handlers`, `ordered-answers`, `log-ordered-echoes`, `held-echoes`. Driver `convex-db-collection/tests/mutation-conformance-driver.ts`, property `adapter-conformance.mutations.convex`. On-demand, source shapes: not started | +| query-db | client cache and refetch, on-demand, mutations, reload and retry, persistence | Not started; not a push source, so the eager family does not apply | +| local-only | local and shared stores, mutations | Not started | +| localStorage | local and shared stores, mutations | Not started | +| Persisted wrapper | persistence wrapper | Not started | +| Storage hosts (8) | storage hosts | Not started | ## Suggested order 1. **Push adapters on the existing family:** RxDB (done), TrailBase (done), PowerSync (done), Electric (eager mode, done), and Convex (eager mode, done). Each needs only a controlled-provider driver. A RED result is a finding against that adapter. 2. **Retire duplicated adapter tests (separate PR, maintainer decision 2026-11-02).** For each adapter on the eager family, classify its existing eager-mode tests and oracles against what the shared oracle generates and observes: remove a test only when the shared family checks the same law, history, path, and observation; narrow one it covers in part; keep the rest. Known overlap: TrailBase's lifecycle oracle and the shared family both own eager startup failure and stream-end health. 3. **The Neon update, run against its driver.** This shows an adapter moving from RED to GREEN. Publishing the package for drivers outside this repository (a public package, `@tanstack/db` as a peer dependency, an exported reach counter) is deferred (maintainer decision 2026-11-02). -4. **The mutations family.** It applies to every adapter except the storage hosts. +4. **The mutations family (in progress).** It applies to every adapter except the storage hosts. 5. **The other families,** one at a time. Each family's new driver work goes to the adapters that declare it. +## Next slice: mutations and confirmation (started 2026-11-02) + +The family checks what an adapter owes when an application writes, through +one shared model and one driver per adapter, as the eager family does. Core +keeps its own owners for laws that no adapter can change: joining a +transaction (MUT-4) and holding synced commits behind pending optimistic +state (MUT-5) stay with `packages/db/tests/optimistic-history-oracle.ts`. + +- **Laws, first cut:** MUT-1 (the write is visible until it settles), MUT-2 + (a rejected write rolls back and `isPersisted` rejects with the error), + MUT-13 and MUT-3 (one handler call per write, with the authored values), + MUT-10 (writes reach the backend in order per Collection), FID-5 and MUT-8 + (after settlement the backend holds the fold of accepted writes; a rejected + write leaves no backend effect), and MUT-7 (under confirmation through sync, + the write stays persisting until its own write arrives in a committed + batch). Authority: the law set's section 9 and FID-5. +- **Capabilities:** `adapter-persistence` (the adapter writes to the backend + itself: RxDB, TrailBase, PowerSync) and `confirm-through-sync` (the + application's handler writes and returns evidence: Electric, Convex). +- **Model:** the synced rows, the backend rows, and the pending writes in + author order with their outcome. Visible rows are the synced rows with the + pending writes folded on; a settled write leaves the fold. +- **Grammar:** single-row inserts, updates, and deletes through the + Collection; the backend accepts or rejects a write, at once or held; the + backend's copy arrives through sync, at once or withheld; a commit by + another client while no write is pending. Manual and multi-mutation + transactions, and commits by another client while a write is pending + (MUT-11), are planned grammar extensions, not generated yet. +- **Observations:** visible rows and change events, each transaction's state + and settlement, the backend rows and the order of backend operations, + handler calls. +- **Calibration:** a reference adapter for each capability and hostile + mutants per law. +- **Order:** the family and its reference adapter, then RxDB, TrailBase, + PowerSync (adapter persistence), then Electric and Convex (confirmation + through sync). + ## Migration log - **2026-10-29, TrailBase, eager live source.** Both campaigns are GREEN. @@ -499,3 +536,150 @@ error/sync-give-up, accept]`). Fix: remember that give-up and fail the replay, or skip the check on the React path, are rejected. A fixed-seed campaign now runs the React client. - **Size:** convex-db-collection +26 gzip bytes for LIFE-10. +- **2026-11-03, mutations family, reference adapter.** The family's oracle + and reference adapter are GREEN on both campaigns, with every declared + reach state met, and seven mutants each rejected with their law. + - **Oracle changes this run exposed:** a publication of unchanged row values + (core's `$hasPendingWrites` update) is not a change of row values; a + subscription forwards changes only for keys it sent, so the recorder + starts from the initial state. + - **Maintainer decision:** a removed field stays an own `undefined`, the + documented rule; the core owner `optimistic-history-oracle` now generates + field removal. +- **2026-11-03, RxDB, mutations.** Both campaigns are GREEN after an + adapter fix, with every declared reach state met. + - **Driver:** a real RxDB database on memory storage; the provider wraps + the storage's `bulkWrite`, stores JSON documents, and rejects with a + per-row write error. An insert the storage refuses as a conflict, which + RxDB's `bulkUpsert` sends before it updates, is not a request. + - **RED on the parent:** 38 of 300 fixed-campaign histories failed MUT-10 + (writes did not wait for the earlier one) and 24 failed MUT-2 (a rejected + delete rejected with an array). Fix: writes run in author order, and a + failed delete rejects with RxDB's `RxError`. The parent is the hostile + mutant for both. + - **Oracle changes this migration exposed:** a driver declares how its + provider stores an authored `undefined` field (`undefinedField`, FID-1's + value domain), and the model holds an accepted write's sync transaction + while another mutation persists (the glossary's held optimistic row). A + JSON-provider reference campaign and an eighth mutant calibrate both. + - **Decided 2026-11-03:** a storage `bulkWrite` that throws wedges the + incremental write queue of RxDB 17.1.0 to 17.5.0, so `incrementalPatch` + never settles (pubkey/rxdb#9025, fixed in 17.6.0). The adapter keeps + `incrementalPatch`; its RxDB dev dependency is now 17.6.0 and its peer + range `>=17.6.0`. A driver can declare `rejectionKinds`; RxDB declares + a per-row write error and a thrown storage error. RED on 17.1.0 (both + campaigns failed HEALTH-5 at a thrown update), GREEN on 17.6.0. + - **Size:** rxdb-db-collection +43 gzip bytes for MUT-10. +- **2026-11-03, mutations family, server rewrites.** A new capability, + `server-rewrites`: an accepted insert's or update's plan can rewrite the + stored row (normalize the title, drop `note`, set `note`), as server + defaults, triggers, and normalization do. The request still carries the + authored row; the provider's copy is the row's value. New reach state + `server-rewrote-copy`, and a grammar arm for updates without `note`. + - **Calibration:** the reference mutant `writes-back-authored` survives + without the capability and fails FID-5 with it. RxDB rewrites through + its `preInsert` and `preSave` hooks; with no-op hooks the fixed campaign + fails FID-5. + - **Result:** the reference adapter and RxDB are GREEN, with no + production change. +- **2026-11-03, mutations family, SQL providers and confirmation through + sync.** New capabilities `field-updates` (an update sends only its changed + fields and the provider merges them, the law set's MUT-8 clause) and + `held-echoes` (the provider answers before its change feed delivers the + copy; the mutation stays pending until it arrives, MUT-7), a `deliver` + step, and `undefinedField: 'null'` (a SQL row: a removed field is sent as + `null`, and a complete row carries every column). A SQL-provider reference + campaign and three mutants calibrate them (a complete-row update, a dropped + removal, settling on the answer). Requests now compare by the fields they + carry. A reach count that credited a rejected write as accepted is fixed. +- **2026-11-03, TrailBase, mutations.** Both campaigns are GREEN after an + adapter fix, with every declared reach state met. + - **RED on the parent:** of 300 fixed-campaign histories, 37 failed MUT-8 + (a removed field was sent as an empty patch), 27 failed MUT-10 (writes + did not wait), and 5 failed MUT-1 (a write was confirmed by an earlier + event on the same key). + - **Fix:** a removed field is sent as `null`; writes run in author order; + a write is confirmed only by an event after it began, recorded only + while a write is admitted, which removes the periodic cleanup timer. + A mutant with the old confirmation fails MUT-1. Refined on 2026-11-04: + same-key histories showed that an earlier write's late event could + still confirm a later write on the key, so the adapter now counts + events per key and confirms the n-th answered write on a key by the + n-th event on it; a late event from an earlier write does not confirm + a later one. + - **Size:** trailbase-db-collection −46 gzip bytes. +- **2026-11-03, PowerSync, mutations.** Both campaigns are GREEN after an + adapter fix, with every declared reach state met. + - **Driver:** a real local PowerSync database is the provider; each + adapter statement on the table is a request, a held echo holds the + change watcher's next callback, and the propagation step waits for + every write's receipt or, while something is held, for quiet. + - **RED on the parent:** both campaigns failed MUT-10: a write started + once the earlier one committed locally, before its diff trigger + observed it. The law set's "decides" is settlement. Fix: each write + runs after the earlier one settles. Removals already reach SQLite as + `NULL`. + - **Oracle change this migration exposed:** `server-rewrote-copy` counted + a copy that differs only by the value domain (a removal under `null`), + so a driver without `server-rewrites` reached an undeclared state. It + now counts only a plan's rewrite. + - **Size:** powersync-db-collection +26 gzip bytes for MUT-10. +- **Decided 2026-11-03, MUT-10 under confirmation through sync.** "The + earlier one decides" is the provider's answer, not the write's + confirmation through sync: core does not chain independent transactions, + and an application that needs that chains them itself. Waiting for + confirmation made a burst of 50 PowerSync inserts take 1.6 s instead of + 31 ms. + - **Oracle:** an answered write is decided, so the next write's request is + owed; it stays pending until its copy arrives. The reference SQL adapter + now starts the next write at the answer and confirms each write by its + own commit; new mutants `chains-on-confirmation` (MUT-8) and + `confirms-by-high-water`, which confirms a withheld copy once a later + one applies (MUT-1). + - **PowerSync:** the ordering change is reverted; the write lock already + orders its writes. Its driver no longer declares `held-writes` or + `held-echoes` (a held statement holds the lock the diff flush needs, and + one watcher callback applies every write). GREEN with no adapter change; + a mutant that settles before its diff trigger observes the write fails + MUT-1. + - **TrailBase:** a write starts after the earlier one's HTTP response, not + after its change event. The committed version, which waited for the + event, fails the fixed campaign. +- **2026-11-03, Electric, mutations.** Both campaigns are GREEN with no + adapter change, with every declared reach state met. + - **Driver:** Electric does not persist writes, so the driver supplies + application handlers that write through a backend API and await + `awaitTxId`; the real `ShapeStream` reads a shape log that carries each + commit's txid in commit order. + - **Oracle changes this migration exposed:** `application-handlers` (the + application's handlers persist writes, so requests go out when made and + MUT-10 is the application's), `log-ordered-echoes` (a withheld copy holds + back later ones), and `ordered-answers` (the provider answers in arrival + order). Writes that settle at one step settle in an order no law fixes, + so the check then compares each row's own value sequence. Reference + campaigns calibrate each: Electric-like and Convex-like providers, and a + provider that breaks each declaration fails (MUT-7, MUT-8, MUT-13). + - **Mutant:** a txid evidence check that accepts any txid fails MUT-1. +- **2026-11-03, Convex, mutations.** Both campaigns are GREEN with no + adapter change, with every declared reach state met. + - **Driver lessons:** a server answers a `Mutation` after the client + registers it, and sends the `MutationResponse` before the transition + that carries the commit: the client completes a mutation only at a later + transition, and drops a response for a request it is not tracking yet. + - **Mutants:** an adapter that does not await the handler fails MUT-1. One + that skips its wait for the source survives these histories; the + cached-listener path it protects is owned by `tests/collection.test.ts` + and the ownership oracle, which reject it. +- **Decided 2026-11-03, RxDB updates merge named fields.** Same-key + histories found that RxDB's `incrementalPatch` keeps a stored field an + update does not name (seed 371775937). RxDB keeps that behavior: the + driver declares `merged-updates`, under which an update's stored row is + its authored row merged into the provider's row. This revises the law + set's complete-row MUT-8 clause for RxDB. An RxDB that replaces the row + fails MUT-8. +- **Decided 2026-11-03, RxDB update of a deleted document.** It rejects + with `UpdateKeyNotFoundError` instead of fulfilling without persisting. +- **Decided 2026-11-03, TrailBase multi-mutation transactions.** Requests + go out one at a time in mutation order and a rejection ends the + transaction; generating multi-mutation transactions, and the law for a + rejection partway through one, are a follow-up for the mutations owner. diff --git a/packages/db-adapter-conformance/src/driver-support.ts b/packages/db-adapter-conformance/src/driver-support.ts new file mode 100644 index 0000000000..0aba765e22 --- /dev/null +++ b/packages/db-adapter-conformance/src/driver-support.ts @@ -0,0 +1,51 @@ +/** + * Support for receiving drivers: code a driver may share, outside the + * oracle's model and check. + */ + +const sleep = (ms: number) => new Promise((resolve) => setTimeout(resolve, ms)) + +export interface QuietSettleOptions { + /** The cut is ready: everything the adapter owes has happened. */ + done: () => boolean + /** + * Something is held, so some writes legitimately wait; the cut is ready + * once `activity` has not changed for `quietMs`. + */ + blocked: () => boolean + /** A value that changes whenever the adapter or the provider acts. */ + activity: () => unknown + /** The adapter has stopped; nothing more will happen. */ + stopped?: () => boolean + timeoutMs?: number + quietMs?: number +} + +/** + * A propagation step for a provider that can hold requests or copies: wait + * until `done`, or, while something is held, until nothing changes for a + * while, then give the client a few turns to apply what it received. + */ +export async function settleWhenQuiet({ + done, + blocked, + activity, + stopped = () => false, + timeoutMs = 2_000, + quietMs = 25, +}: QuietSettleOptions): Promise { + const deadline = Date.now() + timeoutMs + let last = activity() + let quietSince = Date.now() + while (Date.now() < deadline && !stopped()) { + if (done()) break + if (blocked() && Date.now() - quietSince >= quietMs) break + await sleep(2) + const now = activity() + if (now !== last) { + last = now + quietSince = Date.now() + } + } + for (let turn = 0; turn < 3; turn++) await sleep(0) +} diff --git a/packages/db-adapter-conformance/src/eager-live-source-oracle.ts b/packages/db-adapter-conformance/src/eager-live-source-oracle.ts index ff802aeed4..87607dff3e 100644 --- a/packages/db-adapter-conformance/src/eager-live-source-oracle.ts +++ b/packages/db-adapter-conformance/src/eager-live-source-oracle.ts @@ -1870,7 +1870,8 @@ export class ConformanceViolation extends Error { readonly step: number, readonly expected: unknown, readonly actual: unknown, - readonly history: EagerLiveSourceHistory, + // The failing history, of whichever family's grammar. + readonly history: unknown, ) { super( `${law} violated at step ${step} of ${show(history)}: ` + diff --git a/packages/db-adapter-conformance/src/index.ts b/packages/db-adapter-conformance/src/index.ts index 1d1b6a2c74..113e53c791 100644 --- a/packages/db-adapter-conformance/src/index.ts +++ b/packages/db-adapter-conformance/src/index.ts @@ -34,3 +34,32 @@ export { toSqlRow, } from './observed-view' export type { AdapterCollection } from './observed-view' +export { + checkMutationHistory, + mutationHistoryArb, + mutationModel, + mutationProperty, + mutationReachStates, + mutationReachStatesFor, + rewriteRow, + acceptAtOnce, +} from './mutation-oracle' +export { settleWhenQuiet } from './driver-support' +export type { QuietSettleOptions } from './driver-support' +export type { + MutationCapability, + MutationCheckOptions, + MutationDriver, + MutationFixture, + MutationHistory, + MutationProvider, + MutationReachCounter, + MutationReachState, + MutationReceipt, + MutationStep, + MutationWriter, + ServerRewrite, + SettleCut, + WritePlan, + WriteRequest, +} from './mutation-oracle' diff --git a/packages/db-adapter-conformance/src/mutation-oracle.ts b/packages/db-adapter-conformance/src/mutation-oracle.ts new file mode 100644 index 0000000000..f8ab2b9153 --- /dev/null +++ b/packages/db-adapter-conformance/src/mutation-oracle.ts @@ -0,0 +1,1719 @@ +/** + * # Does a sync adapter persist an application's mutations faithfully? + * + * An application writes to a Collection with `insert`, `update`, and + * `delete`. Each write is an optimistic transaction: core overlays its + * optimistic state on the applied synced rows at once, and the adapter's + * mutation function persists it to the provider. This oracle checks the laws + * that every adapter which persists its own writes owes, whatever its + * provider: + * + * - **MUT-1.** While a mutation function runs, the public rows show its + * optimistic state. When it fulfills, no publication shows the earlier + * synced value: the optimistic row drops in the same publication that + * applies the sync transaction carrying the provider's copy of the write. + * While another mutation is persisting, that sync transaction is held and + * the write's optimistic row stays as a held optimistic row until it + * applies (glossary). A one-row write publishes exactly one change of row + * values, and one more when its provider copy differs from the authored + * row, as a JSON provider's does for a removed field. Core also + * publishes an update when only a virtual row field changes, such as + * `$hasPendingWrites` when the write is confirmed; the check compares row + * values only, so such a publication counts as no change here, and the + * virtual-row-field owner (`packages/db/tests/query/virtual-row-fields-oracle.test.ts`) + * judges it. + * - **MUT-2.** When the provider rejects a write, `when('settled')` rejects + * with the provider's error (or, under `matchesFailure`, the value the driver's + * relation names), the optimistic state drops in one publication that is + * exactly its inverse, and other pending mutations stay. + * - **MUT-13, MUT-3, MUT-8.** Each write reaches the provider as exactly one + * request of its type for its key, carrying the authored row (an update + * sends the complete authored row) in the provider's value domain (FID-1): + * a JSON provider has no `undefined`, so a field the update removes is + * absent from its request and its copy, though the optimistic row holds it + * as an own `undefined`. A driver declares this with `undefinedField`. + * - **MUT-10.** Writes are ordered per Collection: a write submitted while an + * earlier one is pending reaches the provider only after the earlier one + * is decided, accepted or rejected by the provider's answer. A write does + * not wait for an earlier one's confirmation through sync: core does not + * chain independent transactions, and an application that needs that + * chains them itself (maintainer decision 2026-11-03). + * - **FID-5.** At a quiescent cut the public rows, the exposed authoritative + * base (`collection.base`), and the provider's rows are identical, and all + * three equal the model: the provider's starting rows with every accepted + * write's copy folded in author order, plus commits from other clients. A + * rejected write leaves no effect in the provider's rows. A provider may + * store a row that differs from the authored one (`server-rewrites`: a + * server default, trigger, or normalization); its copy is then the row's + * value, and the optimistic row gives way to it when the copy applies. + * - **HEALTH-5.** No history causes an unhandled rejection or an undeclared + * error- or warning-level console diagnostic. + * + * Authority: the adapter conformance law set (issue #2118), section 9 and + * FID-5; the optimistic state, held optimistic row, and accepted sync + * transaction terms in `docs/contributing/glossary.md`. Core owns joining a + * transaction (MUT-4) and holding synced commits behind pending optimistic + * state (MUT-5) in `packages/db/tests/optimistic-history-oracle.ts`. + * + * Model terms. A **quiescent cut** is model-only: no mutation function is + * pending, the provider has delivered every change, and every committed sync + * transaction has applied. A **write request** is one call the adapter makes + * to the provider for one mutation. A **plan** is the provider's answer to the + * next write request, which the check hands the provider before the write: it + * accepts or rejects, at once or after a `release`. + * + * Boundary. The oracle checks auto-transaction writes of one row + * (`collection.insert`, `update`, `delete`) on a ready Collection whose + * adapter persists them itself or, with `application-handlers`, confirms the + * writes the application's handlers persist. A write may follow a pending + * one on the same key. With `held-echoes` it checks confirmation through + * sync (MUT-7). It does not check manual or multi-mutation transactions, + * schema validation, or `awaitMatch`. Commits from other clients arrive only + * while no mutation is pending, so MUT-11 is not checked; a commit held + * behind a pending mutation is core's MUT-5. + */ +import fc from 'fast-check' +import { ConformanceViolation } from './eager-live-source-oracle' +import type { + ConformanceRow, + ObservedCollection, + ProviderChange, +} from './eager-live-source-oracle' + +// --------------------------------------------------------------------------- +// Driver contract +// --------------------------------------------------------------------------- + +/** Optional capabilities a driver declares; the grammar uses only these. */ +export type MutationCapability = + /** The provider can reject a write request (MUT-2). */ + | `write-rejection` + /** The provider can hold a write request until `release` (MUT-1, MUT-10). */ + | `held-writes` + /** The provider can commit a change made by another client. */ + | `foreign-commits` + /** + * The provider can store an accepted write's row with its own changes, as + * server defaults, triggers, and normalization do (`ServerRewrite`). + */ + | `server-rewrites` + /** + * Updates send only the changed fields, and the provider merges them into + * its row (the law set's `field-updates`, MUT-8). + */ + | `field-updates` + /** + * The provider can answer an accepted write before its change feed + * delivers the copy, which it then delivers at `deliver`. Declare it only + * for an adapter that confirms writes through sync (MUT-7), and only when + * the provider can withhold each write's copy on its own: later writes + * proceed while a copy is withheld, and `deliver` delivers the oldest. + */ + | `held-echoes` + /** + * The provider's change feed delivers copies in commit order, as a log + * does: a withheld copy also holds back every later commit's copy, and + * `deliver` releases it with the copies queued behind it. + */ + | `log-ordered-echoes` + /** + * The application's own handlers persist each write; the adapter supplies + * only confirmation utilities. The adapter cannot order the handlers, so + * each write's request goes out when it is made: MUT-10 is the + * application's, since core does not chain independent transactions + * (maintainer decision 2026-11-03). + */ + | `application-handlers` + /** + * The provider answers requests in the order they arrive, as a server + * that runs one client's mutations in order does: a request behind a held + * one waits for it. Meaningful with `application-handlers`, whose requests + * go out at once. + */ + | `ordered-answers` + /** + * An update writes the fields its authored row names into the provider's + * row and keeps the others, as RxDB's `incrementalPatch` does; a removed + * field is named, so it is cleared. Its write, as the provider stores it, + * is that merge (maintainer decision 2026-11-03, revising the law set's + * complete-row MUT-8 clause for RxDB). + */ + | `merged-updates` + +/** The optimistic transaction a write call returns. */ +export interface MutationReceipt { + readonly state: string + /** Settles when the mutation function settles, with its outcome. */ + when: (state: `settled`) => Promise +} + +/** The driver's writes, through the adapter's Collection. */ +export interface MutationWriter { + insert: (row: ConformanceRow) => MutationReceipt + /** + * Replace the held row's fields with `row`'s: a complete authored row. A + * `row` without `note` removes a note value the held row holds; a held + * row without one is left as it is. + */ + update: (row: ConformanceRow) => MutationReceipt + delete: (id: number) => MutationReceipt +} + +/** One write request the adapter made to the provider. */ +export interface WriteRequest { + readonly type: `insert` | `update` | `delete` + readonly id: number + /** + * The row the request carries; absent for a delete. Under + * `field-updates`, an update carries only its changed fields. + */ + readonly row?: Readonly> +} + +/** + * A change the provider makes to an accepted insert's or update's row before + * it stores it. Its copy, not the authored row, is then the row's value: + * + * - `none`: the provider stores the row as sent; + * - `retitle`: it normalizes the title to upper case; + * - `drop-note`: it does not store `note`; + * - `set-note`: it sets `note` to `server`, as a server default does. + */ +export type ServerRewrite = `none` | `retitle` | `drop-note` | `set-note` + +/** The row a provider stores for `row` under `rewrite`. */ +export function rewriteRow( + row: ConformanceRow, + rewrite: ServerRewrite, +): ConformanceRow { + switch (rewrite) { + case `none`: + return row + case `retitle`: + return { ...row, title: row.title.toUpperCase() } + case `drop-note`: { + const { note: _dropped, ...rest } = row + return rest + } + case `set-note`: + return { ...row, note: `server` } + } +} + +/** The provider's answer to the next write request. */ +/** The plan for a request the check did not plan: accept it at once. */ +export const acceptAtOnce: WritePlan = { + outcome: `accept`, + held: false, + echoHeld: false, + rewrite: `none`, + error: new Error(`unplanned write`), +} + +export interface WritePlan { + readonly outcome: `accept` | `reject` + /** + * What an accepted insert or update stores (`server-rewrites`); `none` + * otherwise. The request still carries the authored row. + */ + readonly rewrite: ServerRewrite + /** Answer only at `release`, so the write stays pending until then. */ + readonly held: boolean + /** + * After accepting, deliver the copy through the change feed only at + * `deliver` (`held-echoes`). + */ + readonly echoHeld: boolean + /** + * How the provider rejects, one of the driver's `rejectionKinds`; absent + * when the driver declares none. + */ + readonly rejectionKind?: string + /** The error a rejection answers with. */ + readonly error: Error +} + +/** + * The provider a driver controls. It answers write requests by plan, in the + * order they arrive, and delivers every accepted write and every commit from + * another client through the adapter's sync subscription, as a provider's + * change feed does. It records every request; it must not merge, reorder, or + * drop them. + */ +export interface MutationProvider { + /** Queue the answer to the next write request that arrives. */ + plan: (plan: WritePlan) => void + /** Answer the oldest held request by its plan (`held-writes`). */ + release?: () => void | Promise + /** Deliver the oldest withheld copy through the change feed (`held-echoes`). */ + deliver?: () => void | Promise + /** Commit a change made by another client (`foreign-commits`). */ + foreignCommit?: (change: ProviderChange) => void | Promise + /** The provider's committed rows. */ + rows: () => + ReadonlyArray | Promise> + /** Every write request so far, in arrival order. */ + requests: () => ReadonlyArray +} + +/** What the check tells a fixture's propagation step. */ +export interface SettleCut { + /** How many writes' `when('settled')` receipts have not settled yet. */ + unsettled: () => number +} + +export interface MutationFixture { + /** The adapter's Collection, read as conformance rows. */ + collection: ObservedCollection + /** The exposed authoritative base (`collection.base`) as conformance rows. */ + base: () => Iterable + writer: MutationWriter + provider: MutationProvider + /** + * Answers every held request and delivers every withheld copy. The check + * calls it before cleanup, so no write outlives the history waiting on one. + */ + drain?: () => void | Promise + /** The adapter's propagation step, awaited before every comparison. */ + settle?: (cut: SettleCut) => Promise + dispose?: () => void | Promise +} + +export interface MutationDriver { + readonly adapter: string + readonly capabilities: ReadonlySet + /** + * When the adapter reports its own error for a provider rejection, the + * relation that names the value owed (MUT-2's named error). Without it, + * `when('settled')` must reject with the provider's error by identity. + */ + readonly matchesFailure?: (reported: unknown, failure: Error) => boolean + /** + * The ways the provider can reject a write request, such as an error in + * its response and a thrown exception (`write-rejection`). The grammar + * picks one for each rejection, and each must be reached; MUT-2 owes the + * same outcome for every kind. + */ + readonly rejectionKinds?: ReadonlyArray + /** + * How the provider stores an authored field whose value is `undefined`, + * as an update that removes a field authors it. FID-1 parameterizes row + * values by the adapter's value domain: `own` keeps the field as an own + * `undefined`, as a structured clone does; `absent` drops it, as a JSON + * document must, since JSON has no `undefined`; `null` sends it as `null` + * and stores `NULL`, which the provider's rows report as an absent field, + * as a SQL row does. Default `own`. + */ + readonly undefinedField?: `own` | `absent` | `null` + /** Console diagnostics the adapter declares (HEALTH-5). */ + readonly declaredDiagnostic?: ( + level: `error` | `warn`, + args: ReadonlyArray, + ) => boolean + /** A ready-to-preload fixture whose provider starts with `seed`. */ + create: ( + seed: ReadonlyArray, + ) => MutationFixture | Promise +} + +// --------------------------------------------------------------------------- +// History grammar +// --------------------------------------------------------------------------- + +/** + * A history starts with the provider's seed rows, preloads the Collection, + * and applies these steps. The model decides which steps are legal now; an + * illegal step is skipped and counted, so a list stays meaningful after + * shrinking. + * + * - `write`: an insert of a key the public rows lack, or an update or delete + * of a key they hold, also while earlier mutations on that key are + * pending, as long as their plans accept them; an arm generates two writes + * on one key in a row. An update + * that changes no field is not generated. An update without `note` of a + * row that holds one removes the field. Its plan rejects only with + * `write-rejection`, in one of the driver's `rejectionKinds` when it + * declares several, is held only with `held-writes`, and rewrites an + * accepted insert's or update's row only with `server-rewrites`. + * - `release`: the provider answers the oldest held request (`held-writes`). + * - `deliver`: the provider delivers the oldest copy it answered but + * withheld from its change feed (`held-echoes`). + * - `foreign`: another client commits an upsert or a delete while no + * mutation is pending (`foreign-commits`). + * + * After the last step the check releases every held request, reaching a + * quiescent cut. Keys range over 1..4 and titles over three values, so writes + * revisit keys; a row carries `note` half the time, so an update can add or + * remove a field, and a separate arm generates updates without `note` so + * that removals are common. With `server-rewrites`, half of the plans + * rewrite an accepted row. + */ +export type MutationStep = + | { + readonly kind: `write` + readonly type: `insert` | `update` | `delete` + readonly id: number + readonly title: string + readonly note?: string + readonly outcome: `accept` | `reject` + readonly held: boolean + readonly echoHeld: boolean + readonly rewrite: ServerRewrite + /** One of the driver's `rejectionKinds`, for a rejection. */ + readonly rejectionKind?: string + } + | { readonly kind: `release` } + | { readonly kind: `deliver` } + | { readonly kind: `foreign`; readonly change: ProviderChange } + +/** A write step. */ +export type WriteStep = Extract + +export interface MutationHistory { + readonly seed: ReadonlyArray + readonly steps: ReadonlyArray +} + +const rowArb: fc.Arbitrary = fc.record( + { + id: fc.integer({ min: 1, max: 4 }), + title: fc.constantFrom(`a`, `b`, `c`), + note: fc.constantFrom(`x`, `y`), + }, + { requiredKeys: [`id`, `title`] }, +) + +/** Generates histories whose steps use only the driver's capabilities. */ +export function mutationHistoryArb( + domain: Pick, +): fc.Arbitrary { + const { capabilities } = domain + const kinds = domain.rejectionKinds ?? [] + const write = fc + .record({ + type: fc.constantFrom( + `insert` as const, + `update` as const, + `delete` as const, + ), + row: rowArb, + outcome: capabilities.has(`write-rejection`) + ? fc.constantFrom( + `accept` as const, + `accept` as const, + `reject` as const, + ) + : fc.constant(`accept` as const), + held: capabilities.has(`held-writes`) ? fc.boolean() : fc.constant(false), + echoHeld: capabilities.has(`held-echoes`) + ? fc.boolean() + : fc.constant(false), + rewrite: capabilities.has(`server-rewrites`) + ? fc.constantFrom( + `none`, + `none`, + `none`, + `retitle`, + `drop-note`, + `set-note`, + ) + : fc.constant(`none`), + rejectionKind: + kinds.length > 0 + ? fc.constantFrom(...kinds) + : fc.constant(undefined), + }) + .map( + ({ + type, + row, + outcome, + held, + echoHeld, + rewrite, + rejectionKind, + }): WriteStep => ({ + kind: `write`, + type, + id: row.id, + title: row.title, + ...(row.note === undefined ? {} : { note: row.note }), + outcome, + held, + echoHeld: outcome === `accept` && echoHeld, + rewrite: type === `delete` ? `none` : rewrite, + ...(outcome === `reject` && rejectionKind !== undefined + ? { rejectionKind } + : {}), + }), + ) + const steps: Array<{ + weight: number + arbitrary: fc.Arbitrary + }> = [ + { weight: 6, arbitrary: write }, + // An update without `note`: it removes the field when the row holds one. + { + weight: 2, + arbitrary: write.map(({ note: _note, ...step }): WriteStep => ({ + ...step, + type: `update`, + })), + }, + ] + if (capabilities.has(`held-writes`)) { + steps.push({ + weight: 3, + arbitrary: fc.constant({ kind: `release` as const }), + }) + } + if (capabilities.has(`held-echoes`)) { + steps.push({ + weight: 3, + arbitrary: fc.constant({ kind: `deliver` as const }), + }) + } + if (capabilities.has(`foreign-commits`)) { + steps.push({ + weight: 2, + arbitrary: fc.oneof( + rowArb.map((row): MutationStep => ({ + kind: `foreign`, + change: { type: `upsert`, row }, + })), + fc.integer({ min: 1, max: 4 }).map((id): MutationStep => ({ + kind: `foreign`, + change: { type: `delete`, id }, + })), + ), + }) + } + // Two writes on one key in a row, so a write often follows a pending one + // on its key. + const sameKeyPair = fc + .tuple(write, write) + .map(([first, second]): Array => [ + first, + { ...second, id: first.id }, + ]) + // A held write the provider stores with a field the client has not seen, + // then an update of that key that does not name it: a merged update keeps + // the field, a complete-row one drops it. + const unseenFieldPair = + capabilities.has(`server-rewrites`) && capabilities.has(`held-writes`) + ? [ + { + weight: 2, + arbitrary: fc + .tuple(write, write) + .map(([first, second]): Array => { + const { note: _note, ...update } = second + return [ + { + ...first, + type: first.type === `delete` ? `insert` : first.type, + outcome: `accept`, + held: true, + rewrite: `set-note`, + }, + { + ...update, + type: `update`, + id: first.id, + outcome: `accept`, + }, + ] + }), + }, + ] + : [] + return fc.record({ + seed: fc.uniqueArray(rowArb, { selector: (row) => row.id, maxLength: 3 }), + steps: fc + .array( + fc.oneof( + ...steps.map(({ weight, arbitrary }) => ({ + weight, + arbitrary: arbitrary.map((step) => [step]), + })), + { weight: 2, arbitrary: sameKeyPair }, + ...unseenFieldPair, + ), + { minLength: 1, maxLength: 10 }, + ) + .map((groups) => groups.flat()), + }) +} + +// --------------------------------------------------------------------------- +// Independent model +// --------------------------------------------------------------------------- + +/** + * A row as the model compares it: its title, and its note or the note's + * absence. Fields compare by own property, so an own `undefined` note differs + * from an absent one (FID-1). + */ +function rowValue(row: ConformanceRow): string { + return JSON.stringify([ + row.title, + Object.hasOwn(row, `note`) ? [row.note ?? null] : [], + ]) +} + +type RowState = ReadonlyArray<[number, string]> + +function rowState(rows: ReadonlyMap): RowState { + return [...rows].sort(([a], [b]) => a - b) +} + +function rowsOf(rows: Iterable): RowState { + return rowState(new Map([...rows].map((row) => [row.id, rowValue(row)]))) +} + +function sameRows(a: RowState, b: RowState): boolean { + return ( + a.length === b.length && + a.every( + ([id, value], index) => b[index]![0] === id && b[index]![1] === value, + ) + ) +} + +/** + * A request's row as the model compares it: each field it carries other than + * the key, in name order, with `undefined` as `null`, since a request is + * compared by the fields it sends. + */ +function requestValue(fields: Readonly>): string { + return JSON.stringify( + Object.keys(fields) + .filter((field) => field !== `id`) + .sort() + .map((field) => [field, fields[field as keyof ConformanceRow] ?? null]), + ) +} + +/** A model row value back as a row. */ +function rowOf(id: number, value: string): ConformanceRow { + const [title, note] = JSON.parse(value) as [string, Array] + return { + id, + title, + ...(note.length === 0 ? {} : { note: note[0] ?? undefined }), + } +} + +/** A mutation whose plan has not decided it yet. */ +interface PendingWrite { + readonly index: number + readonly type: `insert` | `update` | `delete` + readonly id: number + /** The authored row's model value; absent for a delete. */ + readonly value?: string + /** + * The fields its request carries, in the provider's value domain: the + * authored row, or under `field-updates` an update's changed fields. + * Absent for a delete. + */ + readonly sent?: Readonly> + readonly plan: WritePlan + requested: boolean + /** Accepted, with its copy not yet delivered by the change feed. */ + answered: boolean +} + +type ExpectedSettlement = `pending` | `fulfilled` | { rejected: Error } + +/** + * The model holds only what a legal next step can observe: + * + * - `provider` is the provider's committed rows: the seed, every accepted + * write's copy, and every commit from another client, in commit order. A + * write's copy is the row it sends (an update under `field-updates` + * merges its fields into the provider's row as it is when the write is + * accepted), stored under the driver's `undefinedField` rule and then the + * plan's `ServerRewrite`: after a rewrite the provider's copy, not the + * authored row, is the row's value (FID-5). + * - `rewritten` counts accepted writes whose plan's rewrite made their copy + * differ from their authored row, for reach. A copy can also differ by + * the value domain alone, as a removed field does under `null`. + * - `pending` lists the mutations not yet settled, in author order. MUT-10 + * orders their requests: the first is requested at once, and each later + * one only after every earlier one is decided. A write is decided when + * the provider answers it. An accepted write whose copy the provider + * withholds is `answered`: it is decided, so the next write is + * requested, but it stays pending until its copy arrives (MUT-7). + * - `requests` is every write request MUT-8 and MUT-10 allow so far. + * - `settlements` is each write's `when('settled')` settlement. + * - `applied` is the synced rows core has applied. An accepted write's + * provider copy reaches core as an accepted sync transaction, which is + * held while any optimistic transaction is persisting, so `applied` takes + * `provider` only when no mutation is pending. + * - `heldRows` lists the accepted writes whose sync transaction is held, in + * author order: each keeps its held optimistic row until that transaction + * applies (glossary: accepted sync transaction, held optimistic row). + * + * The public rows are `applied` with `heldRows` and then `pending` folded on + * in author order: core shows a mutation's optimistic state until its + * mutation function settles, and a fulfilled write's provider copy replaces + * its held optimistic row in the publication that applies it. With a + * structured-clone provider the copy equals the authored row, so only a + * provider whose copy differs, such as a JSON one, shows the hold. + */ +interface MutationModel { + /** The driver's `undefinedField` declaration. */ + readonly undefinedField: `own` | `absent` | `null` + /** The driver declares `field-updates`. */ + readonly fieldUpdates: boolean + /** The adapter orders writes (MUT-10): no `application-handlers`. */ + readonly ordered: boolean + /** The driver declares `log-ordered-echoes`. */ + readonly logOrdered: boolean + /** The driver declares `ordered-answers`. */ + readonly answersInOrder: boolean + /** The driver declares `merged-updates`. */ + readonly mergedUpdates: boolean + provider: Map + applied: Map + heldRows: Array + pending: Array + /** Answered writes whose copies are not delivered, in commit order. */ + echoes: Array + /** Every write, by index. */ + writes: Array + rewritten: number + /** Merged updates whose stored row kept a field they did not name. */ + mergeKept: number + requests: Array<{ type: WriteRequest[`type`]; id: number; value?: string }> + settlements: Array +} + +function initialModel( + seed: ReadonlyArray, + domain: Pick, +): MutationModel { + const provider = new Map(seed.map((row) => [row.id, rowValue(row)])) + return { + undefinedField: domain.undefinedField ?? `own`, + fieldUpdates: domain.capabilities.has(`field-updates`), + ordered: !domain.capabilities.has(`application-handlers`), + logOrdered: domain.capabilities.has(`log-ordered-echoes`), + answersInOrder: domain.capabilities.has(`ordered-answers`), + mergedUpdates: domain.capabilities.has(`merged-updates`), + provider, + applied: new Map(provider), + heldRows: [], + pending: [], + echoes: [], + writes: [], + rewritten: 0, + mergeKept: 0, + requests: [], + settlements: [], + } +} + +/** + * The row a write authors. An update that drops `note` from a row holding one + * removes the field, which the update API represents as an own `undefined` + * field (the documented rule in packages/db/tests/proxy-detachment-contract-oracle.test.ts; + * maintainer decision 2026-11-03). + */ +function authoredRow( + step: WriteStep, + over: string | undefined, +): ConformanceRow { + // The row's `note` stays an own `undefined`: removed when it held a + // value, kept when it already was one. + const removesNote = + step.type === `update` && + step.note === undefined && + over !== undefined && + (JSON.parse(over) as [string, Array])[1].length > 0 + return { + id: step.id, + title: step.title, + ...(step.note === undefined + ? removesNote + ? { note: undefined } + : {} + : { note: step.note }), + } +} + +/** + * The fields a request sends for `fields` under the `undefinedField` rule: + * `own` keeps an own `undefined`, `absent` drops it, and `null` sends it as + * `null`. Under `null` a complete row is a SQL row, so it sends every + * column, an absent `note` as `null`; changed fields send only themselves. + */ +function sentFields( + fields: Readonly>, + undefinedField: MutationModel[`undefinedField`], + complete: boolean, +): Partial { + if (undefinedField === `own`) return { ...fields } + if (fields.note !== undefined) return { ...fields } + if (!complete && !Object.hasOwn(fields, `note`)) return { ...fields } + const { note: _removed, ...rest } = fields + return undefinedField === `null` + ? { ...rest, note: null as unknown as undefined } + : rest +} + +/** + * The fields an update changes, as core's `changes` holds them: each authored + * field that differs from the row it was authored over, a removed field as + * an own `undefined`. + */ +function changedFields( + authored: ConformanceRow, + over: ConformanceRow, +): Partial { + return { + id: authored.id, + ...(authored.title === over.title ? {} : { title: authored.title }), + ...(Object.hasOwn(authored, `note`) && + (authored.note !== over.note || !Object.hasOwn(over, `note`)) + ? { note: authored.note } + : {}), + } +} + +/** The provider's copy of an accepted insert or update. */ +function copyOf(state: MutationModel, write: PendingWrite): ConformanceRow { + return rewriteRow(storedFields(state, write), write.plan.rewrite) +} + +/** + * The row an accepted insert or update stores before the provider's own + * rewrite. An update under `field-updates` merges the fields it sends, and + * under `merged-updates` the fields its authored row names, into the + * provider's row; otherwise the sent row replaces it. + */ +function storedFields( + state: MutationModel, + write: PendingWrite, +): ConformanceRow { + const merges = + write.type === `update` && (state.fieldUpdates || state.mergedUpdates) + const base = merges + ? rowOf(write.id, state.provider.get(write.id)!) + : { id: write.id } + const named = state.mergedUpdates + ? rowOf(write.id, write.value!) + : write.sent! + const merged = { ...base, ...named } as ConformanceRow + const { note, ...rest } = merged + // A stored `undefined` or `NULL` is an absent field, except to a + // structured-clone provider, which keeps an own `undefined`. + const kept = + (note ?? undefined) !== undefined || + (state.undefinedField === `own` && Object.hasOwn(merged, `note`)) + return kept ? merged : rest +} + +function fold( + rows: ReadonlyMap, + writes: ReadonlyArray, +): Map { + const folded = new Map(rows) + for (const write of writes) { + if (write.type === `delete`) folded.delete(write.id) + else folded.set(write.id, write.value!) + } + return folded +} + +/** The public rows the model predicts. */ +function visible(state: MutationModel): RowState { + return rowState(fold(fold(state.applied, state.heldRows), state.pending)) +} + +/** + * Decide a write by its plan. An accepted one enters the provider's rows; it + * settles at once unless the provider withholds its copy, and then stays + * pending, answered, until `confirm` (MUT-7). A rejected one settles. + */ +function decide(state: MutationModel, write: PendingWrite): void { + if (write.plan.outcome === `reject`) { + state.settlements[write.index] = { rejected: write.plan.error } + retire(state, write) + return + } + if (write.type === `delete`) state.provider.delete(write.id) + else { + const copy = rowValue(copyOf(state, write)) + state.provider.set(write.id, copy) + if (write.plan.rewrite !== `none` && copy !== write.value) state.rewritten++ + if ( + state.mergedUpdates && + write.type === `update` && + !Object.hasOwn(rowOf(write.id, write.value!), `note`) && + Object.hasOwn(storedFields(state, write), `note`) + ) + state.mergeKept++ + } + // A copy is withheld by its own plan or waits behind an undelivered one: + // any earlier one from a log, an earlier one for its key otherwise, since + // every provider keeps one key's copies in commit order. + if ( + write.plan.echoHeld || + state.echoes.some((earlier) => blocks(state, earlier, write)) + ) { + write.answered = true + state.echoes.push(write) + } else confirm(state, write) +} + +/** + * The oldest withheld copy arrives, and from a log every later copy queued + * behind it, up to the next one its own plan withheld. + */ +function deliverEchoes(state: MutationModel): Array { + confirm(state, state.echoes.shift()!) + const states = [visible(state)] + for (;;) { + const next = state.echoes.find( + (write, index) => + !write.plan.echoHeld && + state.echoes + .slice(0, index) + .every((earlier) => !blocks(state, earlier, write)), + ) + if (!next) return states + state.echoes.splice(state.echoes.indexOf(next), 1) + confirm(state, next) + states.push(visible(state)) + } +} + +/** Whether an undelivered copy holds back a later write's copy. */ +function blocks( + state: MutationModel, + earlier: PendingWrite, + later: PendingWrite, +): boolean { + return state.logOrdered || earlier.id === later.id +} + +/** An accepted write's copy arrives: it fulfills. */ +function confirm(state: MutationModel, write: PendingWrite): void { + state.heldRows.push(write) + state.settlements[write.index] = `fulfilled` + retire(state, write) +} + +function retire(state: MutationModel, write: PendingWrite): void { + state.pending.splice(state.pending.indexOf(write), 1) + // The last persisting transaction settled: every held sync transaction + // applies in its publication. + if (state.pending.length === 0) { + state.applied = new Map(state.provider) + state.heldRows = [] + } +} + +/** + * MUT-10: request the oldest undecided mutation if it has not been; a request + * answered at once is decided at once, which admits the next. A write is + * decided when the provider answers it, also when its copy is withheld + * (maintainer decision 2026-11-03). Returns the public rows after each + * decision, for the publication check. + */ +function advance(state: MutationModel): Array { + const states: Array = [] + for (;;) { + // Ordered, the oldest undecided write goes next; with application + // handlers, every write goes out when it is made, and the provider + // answers each one not held, in arrival order with `ordered-answers`. + const next = state.ordered + ? undecided(state) + : (state.pending.find((write) => !write.requested) ?? + state.pending.find( + (write, index) => + !write.answered && + !write.plan.held && + (!state.answersInOrder || + state.pending + .slice(0, index) + .every((earlier) => earlier.answered)), + )) + if (!next) return states + if (!next.requested) { + next.requested = true + state.requests.push({ + type: next.type, + id: next.id, + ...(next.sent === undefined + ? {} + : { + // A merged update's write is the merge, as the provider + // stores it; nothing commits on its key before it is decided. + value: requestValue( + state.mergedUpdates && next.type === `update` + ? storedFields(state, next) + : next.sent, + ), + }), + }) + if (!state.ordered) continue + } + if (next.plan.held) return states + decide(state, next) + states.push(visible(state)) + } +} + +/** The oldest pending write the provider has not answered. */ +function undecided(state: MutationModel): PendingWrite | undefined { + return state.pending.find((write) => !write.answered) +} + +/** What a step did, for reach and law names. */ +type StepDone = + | { + kind: `write` + type: PendingWrite[`type`] + outcome: WritePlan[`outcome`] + held: boolean + queued: boolean + /** An earlier mutation on the same key was still pending. */ + sameKeyPending: boolean + /** An update that removes `note`, authored as an own `undefined`. */ + removesField: boolean + } + | { kind: `release`; outcome: WritePlan[`outcome`] } + | { kind: `deliver` } + | { kind: `foreign` } + +/** + * Applies one step to the model. Returns undefined when the step is not + * legal now, without moving the model, and otherwise what it did and the + * public rows after each change it publishes, in order. + */ +function stepModel( + state: MutationModel, + step: MutationStep, + plan: (step: WriteStep) => WritePlan, +): { done: StepDone; published: Array } | undefined { + switch (step.kind) { + case `write`: { + // The public rows the write is authored over. + const rows = new Map(visible(state)) + // A write may follow a pending one on the same key: core layers their + // optimistic states in author order. + const sameKey = state.pending.filter((write) => write.id === step.id) + const sameKeyPending = sameKey.length > 0 + // Only behind writes the provider will accept: after a rejected insert + // or delete, what a provider does with the next write on that key is + // the law set's per-adapter `unknownKeyUpdate`, which no driver here + // declares. + if (sameKey.some((write) => write.plan.outcome === `reject`)) + return undefined + // Requests that go out at once can reach the provider out of author + // order, so a write follows one on its key only once that is answered. + if ( + !state.ordered && + !state.answersInOrder && + sameKey.some((write) => !write.answered) + ) + return undefined + const legal = + step.type === `insert` ? !rows.has(step.id) : rows.has(step.id) + const over = rows.get(step.id) + const row = step.type === `delete` ? undefined : authoredRow(step, over) + const value = row && rowValue(row) + // An update that changes no field records no mutation in core; no law + // fixes its persistence, so the grammar leaves it out. + const noop = step.type === `update` && over === value + if (!legal || noop) return undefined + // MUT-10 holds this write back: the adapter orders writes and an earlier + // one waits for its held answer. + const queued = + state.ordered && + state.pending.some((write) => !write.answered && write.plan.held) + const fields = + row && step.type === `update` && state.fieldUpdates + ? changedFields(row, rowOf(step.id, over!)) + : row + const write: PendingWrite = { + index: state.settlements.length, + type: step.type, + id: step.id, + ...(row && { + value, + sent: sentFields(fields!, state.undefinedField, fields === row), + }), + plan: plan(step), + requested: false, + answered: false, + } + state.settlements.push(`pending`) + state.pending.push(write) + state.writes.push(write) + // The optimistic state publishes when the write call returns. + const published = [visible(state), ...advance(state)] + return { + done: { + kind: `write`, + type: step.type, + outcome: step.outcome, + held: step.held, + queued, + sameKeyPending, + // Only a held note value is removed. + removesField: + step.type === `update` && + step.note === undefined && + over !== undefined && + typeof (JSON.parse(over) as [string, Array])[1][0] === + `string`, + }, + published, + } + } + case `release`: { + // The provider answers its oldest held request. + const next = state.pending.find( + (write) => !write.answered && write.requested && write.plan.held, + ) + if (!next) return undefined + decide(state, next) + const published = [visible(state), ...advance(state)] + return { + done: { kind: `release`, outcome: next.plan.outcome }, + published, + } + } + case `deliver`: { + if (state.echoes.length === 0) return undefined + const published = [...deliverEchoes(state), ...advance(state)] + return { done: { kind: `deliver` }, published } + } + case `foreign`: { + if (state.pending.length > 0) return undefined + if (step.change.type === `upsert`) + state.provider.set(step.change.row.id, rowValue(step.change.row)) + else state.provider.delete(step.change.id) + state.applied = new Map(state.provider) + return { done: { kind: `foreign` }, published: [visible(state)] } + } + } +} + +/** The model state a step starts from, for the settlement check. */ +interface StepStart { + readonly pending: ReadonlyArray + readonly heldRows: ReadonlyArray + readonly applied: ReadonlyMap + readonly writes: number +} + +function stepStart(state: MutationModel): StepStart { + return { + pending: [...state.pending], + heldRows: [...state.heldRows], + applied: new Map(state.applied), + writes: state.writes.length, + } +} + +/** + * The writes a step settled, and each row's values while they settle in any + * order: the public rows with any subset of them settled. Settlements at one + * step are in no order a law fixes, so a publication may show any of these. + */ +function settlementValues( + state: MutationModel, + start: StepStart, +): { settled: number; values: Map> } { + const pending = [...start.pending, ...state.writes.slice(start.writes)] + const settled = pending.filter( + (write) => state.settlements[write.index] !== `pending`, + ) + const values = new Map>() + for (let mask = 0; mask < 1 << settled.length; mask++) { + const done = new Set(settled.filter((_, bit) => mask & (1 << bit))) + const still = pending.filter((write) => !done.has(write)) + // Held rows and the applied rows change only when nothing is pending. + const rows = + still.length === 0 + ? state.applied + : fold( + fold(start.applied, [ + ...start.heldRows, + ...pending.filter( + (write) => + done.has(write) && + state.settlements[write.index] === `fulfilled`, + ), + ]), + still, + ) + const ids = new Set([...rows.keys(), ...start.applied.keys()]) + for (const id of [...ids, ...pending.map((write) => write.id)]) { + const set = values.get(id) ?? new Set() + set.add(rows.get(id) ?? null) + values.set(id, set) + } + } + return { settled: settled.length, values } +} + +/** + * The publications a step owes: each change of the public rows, in order. A + * change that leaves the rows as they were publishes nothing, as an accepted + * write whose provider copy equals its optimistic state does. + */ +function owedPublications( + before: RowState, + states: ReadonlyArray, +): Array { + const owed: Array = [] + let last = before + for (const next of states) { + if (!sameRows(next, last)) owed.push(next) + last = next + } + return owed +} + +// --------------------------------------------------------------------------- +// Refinement check +// --------------------------------------------------------------------------- + +/** Declared states whose laws need generated histories to reach them. */ +export const mutationReachStates = [ + `insert-accepted`, + `update-accepted`, + `delete-accepted`, + `write-rejected`, + `held-write-pending`, + `write-queued-behind-held`, + `held-write-rejected`, + `foreign-commit`, + `field-removed`, + `same-key-pending`, + `held-optimistic-row`, + `server-rewrote-copy`, + `merge-kept-field`, + `echo-held`, + `echo-queued-behind-withheld`, + `quiescent-after-writes`, +] as const + +/** A rejection of one of the driver's `rejectionKinds`, per kind. */ +export type MutationReachState = + (typeof mutationReachStates)[number] | `rejected-as-${string}` + +/** + * The capabilities a declared state needs: every one in `all`, at least one + * in `any`, and none in `none`. A state with no entry needs none. + */ +const stateNeeds: Partial< + Record< + MutationReachState, + { + all?: ReadonlyArray + any?: ReadonlyArray + none?: ReadonlyArray + } + > +> = { + 'write-rejected': { all: [`write-rejection`] }, + 'held-write-pending': { all: [`held-writes`] }, + // Only an adapter that orders writes holds one back (MUT-10). + 'write-queued-behind-held': { + all: [`held-writes`], + none: [`application-handlers`], + }, + 'held-write-rejected': { all: [`held-writes`, `write-rejection`] }, + 'foreign-commit': { all: [`foreign-commits`] }, + 'held-optimistic-row': { all: [`held-writes`] }, + 'server-rewrote-copy': { all: [`server-rewrites`] }, + 'echo-held': { all: [`held-echoes`] }, + 'merge-kept-field': { all: [`merged-updates`] }, + 'echo-queued-behind-withheld': { all: [`held-echoes`] }, + // A write stays pending past its step only when held or confirmed late. + 'same-key-pending': { any: [`held-writes`, `held-echoes`] }, +} + +/** The declared states a driver's capabilities let histories reach. */ +export function mutationReachStatesFor( + domain: Pick, +): Array { + const has = (capability: MutationCapability) => + domain.capabilities.has(capability) + const kinds = has(`write-rejection`) + ? (domain.rejectionKinds ?? []).map( + (kind): MutationReachState => `rejected-as-${kind}`, + ) + : [] + return [ + ...mutationReachStates.filter((state) => { + const { all = [], any, none = [] } = stateNeeds[state] ?? {} + return all.every(has) && (any?.some(has) ?? true) && !none.some(has) + }), + ...kinds, + ] +} + +type Settlement = `pending` | `fulfilled` | { rejected: unknown } + +function recordSettlement(promise: Promise): { current: Settlement } { + const record: { current: Settlement } = { current: `pending` } + promise.then( + () => { + record.current = `fulfilled` + }, + (error: unknown) => { + record.current = { rejected: error } + }, + ) + return record +} + +const sleep = (ms: number) => new Promise((resolve) => setTimeout(resolve, ms)) + +/** + * The checkpoint premise: an adapter answers its provider and applies the + * provider's change within a few event-loop turns, or within the + * propagation step its fixture declares. A slower adapter shows as a + * mismatch at this cut, not as a pass. + */ +async function settle(fixture: MutationFixture, cut: SettleCut): Promise { + for (let turn = 0; turn < 3; turn++) await sleep(0) + await fixture.settle?.(cut) +} + +/** + * Records the public rows after each change publication (MUT-1, MUT-2). A + * subscription forwards changes only for keys it has sent, so it starts with + * the initial state, which is the baseline and not a publication. + */ +function recordPublications(collection: ObservedCollection) { + const rows = new Map() + let inconsistent = false + let initial = true + let publications: Array = [] + const subscription = collection.subscribeChanges( + (changes) => { + if (initial) { + initial = false + for (const change of changes) + rows.set(change.key, rowValue(change.value)) + return + } + if (changes.length === 0) return + for (const change of changes) { + if (change.type === `delete`) { + if (!rows.delete(change.key)) inconsistent = true + } else { + if ((change.type === `insert`) === rows.has(change.key)) + inconsistent = true + rows.set(change.key, rowValue(change.value)) + } + } + publications.push( + inconsistent ? [[-1, `inconsistent change events`]] : rowState(rows), + ) + }, + { includeInitialState: true }, + ) + // An empty Collection's initial state is no callback at all. + initial = false + return { + take: () => { + const taken = publications + publications = [] + return taken + }, + stop: () => subscription.unsubscribe(), + } +} + +function watchDetachedFailures(driver: MutationDriver) { + const found: Array = [] + const onRejection = (reason: unknown) => { + found.push(`unhandled rejection: ${String(reason)}`) + } + const hasProcess = typeof process !== `undefined` + if (hasProcess) process.on(`unhandledRejection`, onRejection) + const original = { error: console.error, warn: console.warn } + for (const level of [`error`, `warn`] as const) { + console[level] = (...args: Array) => { + if (!driver.declaredDiagnostic?.(level, args)) { + found.push(`console.${level}: ${args.map(String).join(` `)}`) + } + } + } + return { + found: () => found, + stop: () => { + if (hasProcess) process.off(`unhandledRejection`, onRejection) + console.error = original.error + console.warn = original.warn + }, + } +} + +export interface MutationCheckOptions { + /** Called when the history enters a declared state (ORC-015). */ + reach?: (state: MutationReachState) => void +} + +/** + * Runs one history through a driver and compares every checkpoint with the + * model. Throws a `ConformanceViolation` naming the first violated law. The + * fixture is always cleaned up; a cleanup failure after a violation is kept + * as a secondary error (ORC-010). + */ +export async function checkMutationHistory( + driver: MutationDriver, + history: MutationHistory, + options: MutationCheckOptions = {}, +): Promise<{ accepted: number; skipped: number }> { + const reach = options.reach ?? (() => {}) + const detached = watchDetachedFailures(driver) + let created: MutationFixture | undefined + try { + created = await driver.create(structuredClone(history.seed)) + } catch (error) { + detached.stop() + throw error + } + const fixture = created + const { collection, provider, writer } = fixture + const state = initialModel(history.seed, driver) + const receipts: Array<{ current: Settlement }> = [] + const cut: SettleCut = { + unsettled: () => + receipts.filter((receipt) => receipt.current === `pending`).length, + } + let accepted = 0 + let skipped = 0 + let primary: unknown + let publications: ReturnType | undefined + + const steps = history.steps + const violation = (law: string, step: number, want: unknown, got: unknown) => + new ConformanceViolation(law, step, want, got, history) + const matches = (reported: unknown, failure: Error) => + driver.matchesFailure + ? driver.matchesFailure(reported, failure) + : reported === failure + + /** MUT-1, MUT-2, MUT-8, MUT-10, MUT-13, FID-5, HEALTH-5 at one cut. */ + const compare = async (stepIndex: number) => { + await settle(fixture, cut) + const found = detached.found() + if (found.length > 0) + throw violation(`HEALTH-5`, stepIndex, `no detached failure`, found) + // The provider's request log first: it observes MUT-8, MUT-10, and + // MUT-13 directly, where the rows show their effect only later. + const requests = provider.requests().map((request) => ({ + type: request.type, + id: request.id, + ...(request.row === undefined + ? {} + : { value: requestValue(request.row) }), + })) + if (JSON.stringify(requests) !== JSON.stringify(state.requests)) + throw violation( + requests.length > state.requests.length + ? `MUT-10/MUT-13 (provider requests)` + : `MUT-8 (provider requests)`, + stepIndex, + state.requests, + requests, + ) + const quiescent = state.pending.length === 0 + const rows = rowsOf(collection.values()) + if (!sameRows(rows, visible(state))) + throw violation( + quiescent ? `FID-5 (public rows)` : `MUT-1 (public rows)`, + stepIndex, + visible(state), + rows, + ) + const answered = new Set( + state.pending + .filter((write) => write.answered) + .map((write) => write.index), + ) + state.settlements.forEach((expected, index) => { + const got = receipts[index]!.current + const ok = + expected === `pending` || expected === `fulfilled` + ? got === expected + : typeof got === `object` && matches(got.rejected, expected.rejected) + if (!ok) + throw violation( + typeof expected === `object` + ? `MUT-2 (settlement)` + : answered.has(index) + ? `MUT-7 (settlement)` + : `MUT-13 (settlement)`, + stepIndex, + typeof expected === `object` ? expected.rejected.message : expected, + typeof got === `object` ? String(got.rejected) : got, + ) + }) + if (!quiescent) return + // FID-5 at a quiescent cut: public rows, base, and provider are one. + const base = rowsOf(fixture.base()) + if (!sameRows(base, rowState(state.provider))) + throw violation(`FID-5 (base)`, stepIndex, rowState(state.provider), base) + const providerRows = rowsOf(await provider.rows()) + if (!sameRows(providerRows, rowState(state.provider))) + throw violation( + `FID-5 (provider rows)`, + stepIndex, + rowState(state.provider), + providerRows, + ) + if (receipts.length > 0) reach(`quiescent-after-writes`) + } + + let published = visible(state) + const checkPublications = ( + stepIndex: number, + owed: ReadonlyArray, + law: string, + from: StepStart, + ) => { + // Each change of row values, in order; a publication of the same row + // values (a virtual row field change) is not one. + const start = published + const got: Array = [] + for (const rows of publications!.take()) { + if (!sameRows(rows, got.at(-1) ?? published)) got.push(rows) + } + published = got.at(-1) ?? published + // Writes that settle at one step settle in an order no law fixes. Then + // each row may show only a value it has with some of them settled, and + // changes at most once per settled write and once for the step's own + // optimistic state; an earlier synced value or an extra change still + // shows. + const { settled, values } = settlementValues(state, from) + const ok = + settled > 1 + ? [...values].every(([id, allowed]) => { + const path = [start, ...got].map( + (rows) => rows.find(([key]) => key === id)?.[1] ?? null, + ) + const changes = path.filter( + (value, index) => index > 0 && value !== path[index - 1], + ).length + return ( + path.every((value) => allowed.has(value)) && + changes <= settled + 1 + ) + }) && + [...got.flatMap((rows) => rows.map(([id]) => id))].every((id) => + values.has(id), + ) + : got.length === owed.length && + got.every((rows, index) => sameRows(rows, owed[index]!)) + if (!ok) throw violation(`${law} (publication)`, stepIndex, owed, got) + } + + let rewritten = 0 + /** The declared states the model's state shows after a step. */ + const reachModelStates = () => { + if (state.heldRows.length > 0) reach(`held-optimistic-row`) + if (state.rewritten > rewritten) reach(`server-rewrote-copy`) + if (state.mergeKept > 0) reach(`merge-kept-field`) + if (state.pending.some((write) => write.answered)) reach(`echo-held`) + if (state.echoes.some((write) => !write.plan.echoHeld)) + reach(`echo-queued-behind-withheld`) + rewritten = state.rewritten + } + // The plan the model decided the last write by; the provider gets it too. + let lastPlan: WritePlan | undefined + const planFor = (step: WriteStep): WritePlan => { + // Indexed by write, so a driver's `matchesFailure` can name it; the write + // records its receipt after its plan. + const error = new Error(`conformance write rejected ${receipts.length}`) + lastPlan = { + outcome: step.outcome, + held: step.held, + echoHeld: step.echoHeld, + rewrite: step.rewrite, + ...(step.rejectionKind === undefined + ? {} + : { rejectionKind: step.rejectionKind }), + error, + } + return lastPlan + } + + try { + await collection.preload() + publications = recordPublications(collection) + await compare(-1) + for (const [index, step] of steps.entries()) { + const before = visible(state) + const from = stepStart(state) + const result = stepModel(state, step, planFor) + if (result === undefined) { + skipped++ + continue + } + accepted++ + const { done } = result + let law = `MUT-1` + switch (done.kind) { + case `write`: { + const write = step as WriteStep + provider.plan(lastPlan!) + const row: ConformanceRow = { + id: write.id, + title: write.title, + ...(write.note === undefined ? {} : { note: write.note }), + } + const receipt = + write.type === `insert` + ? writer.insert(row) + : write.type === `update` + ? writer.update(row) + : writer.delete(write.id) + receipts.push(recordSettlement(receipt.when(`settled`))) + if (write.held) reach(`held-write-pending`) + if (done.queued) reach(`write-queued-behind-held`) + if (write.outcome === `reject`) { + reach(`write-rejected`) + if (write.rejectionKind !== undefined) + reach(`rejected-as-${write.rejectionKind}`) + } else if (!write.held) reach(`${write.type}-accepted` as const) + if (done.removesField && write.outcome === `accept`) + reach(`field-removed`) + if (done.sameKeyPending) reach(`same-key-pending`) + if (write.outcome === `reject` && !write.held) law = `MUT-2` + break + } + case `release`: + await provider.release!() + if (done.outcome === `reject`) { + reach(`held-write-rejected`) + law = `MUT-2` + } + break + case `deliver`: + await provider.deliver!() + break + case `foreign`: + await provider.foreignCommit!( + (step as Extract).change, + ) + reach(`foreign-commit`) + break + } + reachModelStates() + await compare(index) + checkPublications( + index, + owedPublications(before, result.published), + law, + from, + ) + } + // Answer every held request and deliver every withheld copy, reaching a + // quiescent cut (FID-5). + while (state.pending.length > 0) { + const before = visible(state) + const from = stepStart(state) + const step: MutationStep = undecided(state) + ? { kind: `release` } + : { kind: `deliver` } + const result = stepModel(state, step, planFor) + if (result === undefined) break + reachModelStates() + if (step.kind === `deliver`) await provider.deliver!() + else await provider.release!() + await compare(steps.length) + checkPublications( + steps.length, + owedPublications(before, result.published), + `MUT-1`, + from, + ) + } + } catch (error) { + primary = error + } + + const cleanupErrors: Array = [] + try { + publications?.stop() + await fixture.drain?.() + await collection.cleanup() + await settle(fixture, cut) + if (primary === undefined && detached.found().length > 0) { + primary = violation( + `HEALTH-5`, + steps.length, + `no detached failure`, + detached.found(), + ) + } + } catch (error) { + cleanupErrors.push(error) + } finally { + detached.stop() + } + try { + await fixture.dispose?.() + } catch (error) { + cleanupErrors.push(error) + } + if (primary !== undefined) { + if (cleanupErrors.length === 0) throw primary + throw new AggregateError(cleanupErrors, `conformance violation`, { + cause: primary, + }) + } + if (cleanupErrors.length > 0) throw cleanupErrors[0] + return { accepted, skipped } +} + +/** A reach counter, such as the one `createOracleReach` returns. */ +export interface MutationReachCounter { + beginHistory: () => void + reach: (state: MutationReachState) => void +} + +/** + * The generated property every driver runs. Shrinking keeps the first + * violated law (ORC-010): a smaller history that fails a different law + * counts as passing. + */ +export function mutationProperty( + driver: MutationDriver, + reach?: MutationReachCounter, +) { + let target: string | undefined + return fc.asyncProperty(mutationHistoryArb(driver), async (history) => { + reach?.beginHistory() + try { + await checkMutationHistory(driver, history, { reach: reach?.reach }) + } catch (error) { + const law = + error instanceof ConformanceViolation + ? error.law + : `failure outside the comparison` + target ??= law + if (law === target) throw error + } + }) +} + +// Exported for calibration tests: the model's prediction for a history. +export const mutationModel = { + initial: initialModel, + step: stepModel, + visible, + owedPublications, +} diff --git a/packages/db-adapter-conformance/tests/reference-mutation-adapter.ts b/packages/db-adapter-conformance/tests/reference-mutation-adapter.ts new file mode 100644 index 0000000000..d94484d3f7 --- /dev/null +++ b/packages/db-adapter-conformance/tests/reference-mutation-adapter.ts @@ -0,0 +1,543 @@ +/** + * A reference adapter that persists its own writes, written to conform to the + * mutation laws, and its production driver. It calibrates the check: the + * adapter must pass, and each hostile mutant must be rejected with the law it + * breaks (ORC-006). + * + * The adapter's sync run loads the provider's rows, marks ready, and applies + * the provider's change feed. Its mutation functions send each mutation to + * the provider one write at a time per Collection (MUT-10). + * + * The provider has three forms: + * + * - by default it stores structured clones and replaces a row on every + * write. An accepted write reaches the change feed before the provider + * answers, so the mutation function returns once the provider answers: + * its sync transaction is committed before the mutation function settles, + * and core drops the optimistic row in the same publication that applies + * it (MUT-1); + * - a JSON provider (`json`) stores and feeds JSON, so a field whose value is + * `undefined` is absent (`undefinedField: 'absent'`); + * - a SQL provider (`sql`), like TrailBase's or PowerSync's service, stores + * rows with every column and an absent field as `NULL`, which its rows and + * change feed report as absent (`undefinedField: 'null'`). An update sends + * only its changed fields, a removed one as `null`, and the provider merges + * them into its row (`field-updates`). The provider can answer before its + * change feed delivers the copy (`held-echoes`), so the adapter confirms + * each write through sync: its mutation function returns when its own + * sync run has applied that write's copy (MUT-7). The next write starts + * when the provider answers, without waiting for that confirmation + * (MUT-10). + */ +import { createCollection } from '@tanstack/db' +import { acceptAtOnce, observedView, rewriteRow } from '../src' +import type { + ConformanceRow, + MutationCapability, + MutationDriver, + MutationFixture, + MutationProvider, + ProviderChange, + WritePlan, + WriteRequest, +} from '../src' + +/** Wrong designs the calibration tests inject, one decision each. */ +export type ReferenceMutationMutant = + /** MUT-1: the provider's copy applies after the mutation function settles. */ + | `applies-copy-late` + /** MUT-2: a rejected write resolves, so its optimistic state stays. */ + | `swallows-rejection` + /** MUT-2: a rejection reports a different error value. */ + | `wraps-rejection` + /** MUT-13: an insert reaches the provider twice. */ + | `writes-insert-twice` + /** MUT-10: a write does not wait for the earlier one to be decided. */ + | `writes-concurrently` + /** FID-5: a delete never reaches the provider. */ + | `skips-delete` + /** MUT-8: an update sends only the changed fields. */ + | `sends-changes-only` + /** + * FID-5: the change feed carries the authored row instead of the stored + * copy, so a JSON provider's removed field stays an own `undefined`. + */ + | `echoes-authored-row` + /** + * FID-5: after the provider accepts, the adapter writes the authored row + * into its synced rows, over the provider's rewritten copy. + */ + | `writes-back-authored` + /** MUT-8 (SQL): an update sends the complete row, not its changed fields. */ + | `sends-complete-row` + /** + * MUT-8 (SQL): an update sends its changes as JSON as they are, so a + * removed field, an own `undefined`, never reaches the provider. + */ + | `drops-removal` + /** + * MUT-7 (SQL): the mutation function returns when the provider answers, + * before its copy arrives through sync. + */ + | `settles-on-answer` + /** + * MUT-10 (SQL): a write waits for the earlier one's confirmation through + * sync, not only for the provider's answer, so it reaches the provider + * late. + */ + | `chains-on-confirmation` + /** + * MUT-7 (SQL): a write counts as confirmed once any later commit's copy + * has applied, so a withheld copy is skipped. + */ + | `confirms-by-high-water` + +export interface ReferenceMutationOptions { + /** A JSON provider. */ + json?: boolean + /** A SQL provider, with field updates and confirmation through sync. */ + sql?: boolean + /** + * The provider can answer before its change feed delivers the copy, so + * the adapter confirms each write through sync (`held-echoes`); a SQL + * provider implies it. + */ + confirms?: boolean + /** The change feed is a log: a withheld copy holds back later ones. */ + log?: boolean + /** The application's handlers persist writes: the adapter orders none. */ + application?: boolean + /** The provider answers requests in arrival order. */ + inOrder?: boolean +} + +const jsonClone = (value: T): T => JSON.parse(JSON.stringify(value)) as T + +/** A provider that answers writes by plan and feeds every commit back. */ +export class ControlledMutationProvider implements MutationProvider { + private readonly committed = new Map() + private readonly plans: Array = [] + private readonly held: Array<() => void> = [] + /** Copies not yet fed, in commit order; `own` marks a plan's withholding. */ + private readonly withheld: Array<{ + feed: () => void + own: boolean + id: number + }> = [] + private readonly log: Array = [] + private listener: + ((change: ProviderChange, sequence: number) => void) | undefined + private commits = 0 + + constructor( + seed: ReadonlyArray, + private readonly options: ReferenceMutationOptions, + private readonly echoesAuthored = false, + ) { + for (const row of seed) this.committed.set(row.id, this.stored(row)) + } + + /** A row as this provider stores it. */ + private stored(row: Partial): ConformanceRow { + if (!this.options.json && !this.options.sql) + return structuredClone(row) as ConformanceRow + const stored = jsonClone(row) as Omit & { + note?: string | null + } + // A SQL column holds `NULL`, which the provider reports as absent. + if (stored.note === null) delete stored.note + return stored as ConformanceRow + } + + // The adapter's side: its change feed, initial read, and writes. + + subscribe( + listener: (change: ProviderChange, sequence: number) => void, + ): () => void { + this.listener = listener + return () => { + if (this.listener === listener) this.listener = undefined + } + } + + snapshot(): Array { + return [...this.committed.values()].map((row) => structuredClone(row)) + } + + /** + * Answers with the commit's sequence number, which its change feed + * carries, so an adapter can confirm the write through sync. + */ + write(request: WriteRequest): Promise { + const { json, sql } = this.options + this.log.push(json || sql ? jsonClone(request) : structuredClone(request)) + const plan = this.plans.shift() ?? acceptAtOnce + return new Promise((resolve, reject) => { + const answer = () => { + if (plan.outcome === `reject`) { + reject(plan.error) + return + } + if (request.type === `delete`) { + resolve( + this.commit({ type: `delete`, id: request.id }, plan.echoHeld), + ) + return + } + // A SQL update merges the fields it sends into the stored row. + const row = + request.type === `update` && sql + ? { ...this.committed.get(request.id), ...jsonClone(request.row) } + : request.row + resolve( + this.commit( + { + type: `upsert`, + row: rewriteRow(row as ConformanceRow, plan.rewrite), + }, + plan.echoHeld, + ), + ) + } + // In order, an answer waits behind every earlier unanswered request. + const entry = { answer, held: plan.held } + this.unanswered.push(entry) + if (plan.held) this.held.push(() => this.answerEntry(entry)) + else if (!this.options.inOrder || this.unanswered[0] === entry) + this.answerEntry(entry) + }) + } + + /** Requests not yet answered, in arrival order. */ + private readonly unanswered: Array<{ answer: () => void; held: boolean }> = [] + + private answerEntry(entry: { answer: () => void; held: boolean }): void { + this.unanswered.splice(this.unanswered.indexOf(entry), 1) + entry.answer() + // In order, the requests behind it that are not held follow. + while ( + this.options.inOrder && + this.unanswered[0] && + !this.unanswered[0].held + ) + this.answerEntry(this.unanswered[0]) + } + + /** + * Commits a change and feeds it back, at once or at `deliver`. Without + * field updates a write replaces the whole row, as a complete-row API does. + */ + private commit(change: ProviderChange, withhold = false): number { + if (change.type === `delete`) this.committed.delete(change.id) + else this.committed.set(change.row.id, this.stored(change.row)) + const sequence = ++this.commits + const fed = this.echoesAuthored + ? structuredClone(change) + : change.type === `delete` + ? change + : { ...change, row: this.stored(change.row) } + const feed = () => this.listener?.(fed, sequence) + // A copy waits behind a withheld one: any, from a log, or one for its + // key, since a provider keeps one key's copies in commit order. + const id = change.type === `delete` ? change.id : change.row.id + if ( + withhold || + this.withheld.some((entry) => this.options.log || entry.id === id) + ) + this.withheld.push({ feed, own: withhold, id }) + else feed() + return sequence + } + + // The oracle's side. + + plan(plan: WritePlan): void { + this.plans.push(plan) + } + + release(): void { + this.held.shift()?.() + } + + deliver(): void { + this.withheld.shift()?.feed() + for (;;) { + const index = this.withheld.findIndex( + (entry, at) => + !entry.own && + this.withheld + .slice(0, at) + .every((earlier) => !this.options.log && earlier.id !== entry.id), + ) + if (index < 0) return + this.withheld.splice(index, 1)[0]!.feed() + } + } + + foreignCommit(change: ProviderChange): void { + this.commit(change) + } + + rows(): ReadonlyArray { + return this.snapshot() + } + + requests(): ReadonlyArray { + return this.log + } +} + +let sequence = 0 + +/** Builds the reference adapter's Collection over `provider`. */ +function referenceCollection( + provider: ControlledMutationProvider, + mutant: ReferenceMutationMutant | undefined, + { + sql = false, + confirms = sql, + application = false, + }: ReferenceMutationOptions, +) { + // MUT-10: each write waits for the earlier one to be decided, by the + // provider's answer. A write's turn covers sending it; its confirmation + // (SQL) follows after the turn. + let turn: Promise = Promise.resolve() + const persist = async ( + sendAll: () => Promise>, + ): Promise => { + if (mutant === `writes-concurrently` || application) { + await confirmAll(await sendAll()) + return + } + const sent = turn.then(sendAll) + const chained = + mutant === `chains-on-confirmation` ? sent.then(confirmAll) : sent + turn = chained.catch(() => undefined) + await confirmAll(await sent) + } + const answer = async (sent: Promise): Promise => { + try { + await sent + } catch (error) { + if (mutant === `swallows-rejection`) return + if (mutant === `wraps-rejection`) + throw new Error(`wrapped`, { cause: error }) + throw error + } + } + // MUT-7 (SQL): a write is confirmed when this sync run has applied the + // commit its answer names. + // Copies can arrive out of commit order, since a withheld one trails later + // ones, so each write waits for its own commit, not for the highest seen. + const applied = new Set() + let highest = 0 + const confirmations = new Map void>() + const isApplied = (commit: number) => + mutant === `confirms-by-high-water` + ? highest >= commit + : applied.has(commit) + const confirmed = (commit: number): Promise => { + if (!confirms || mutant === `settles-on-answer` || isApplied(commit)) + return Promise.resolve() + return new Promise((resolve) => confirmations.set(commit, resolve)) + } + const confirmAll = (commits: Array) => + Promise.all(commits.map(confirmed)).then(() => undefined) + // The `writes-back-authored` mutant writes through the sync run's apply. + let applyChange: ((change: ProviderChange) => void) | undefined + const writeBack = (row: ConformanceRow) => { + if (mutant === `writes-back-authored`) + applyChange?.({ type: `upsert`, row: structuredClone(row) }) + } + /** The row an update request carries. */ + const updateRow = (mutation: { + key: number + modified: ConformanceRow + changes: Partial + }): Partial => { + if (sql && mutant !== `sends-complete-row`) { + const changes: Record = { + id: mutation.key, + ...mutation.changes, + } + // A removed field is `NULL` in its column. + if (mutant !== `drops-removal`) + for (const [field, value] of Object.entries(changes)) + if (value === undefined) changes[field] = null + return changes as Partial + } + return mutant === `sends-changes-only` + ? { id: mutation.key, ...mutation.changes } + : structuredClone(mutation.modified) + } + const collection = createCollection({ + id: `reference-mutations-${++sequence}`, + getKey: (row) => row.id, + sync: { + rowUpdateMode: `full`, + sync: ({ begin, write, commit, markReady }) => { + const keys = new Set() + const apply = (change: ProviderChange) => { + begin() + if (change.type === `delete`) { + if (keys.delete(change.id)) + write({ type: `delete`, key: change.id }) + } else { + write({ + type: keys.has(change.row.id) ? `update` : `insert`, + value: change.row, + }) + keys.add(change.row.id) + } + void commit() + } + begin() + for (const row of provider.snapshot()) { + write({ type: `insert`, value: row }) + keys.add(row.id) + } + void commit() + markReady() + applyChange = apply + const receive = (change: ProviderChange, commitSequence: number) => { + apply(change) + applied.add(commitSequence) + highest = Math.max(highest, commitSequence) + for (const [commit, resolve] of confirmations) + if (isApplied(commit)) { + confirmations.delete(commit) + resolve() + } + } + return provider.subscribe((change, commitSequence) => { + if (mutant === `applies-copy-late`) + setTimeout(() => receive(change, commitSequence), 0) + else receive(change, commitSequence) + }) + }, + }, + onInsert: ({ transaction }) => + answer( + persist(async () => { + const commits: Array = [] + for (const mutation of transaction.mutations) { + const request = { + type: `insert` as const, + id: mutation.key, + // A SQL row has every column: an absent field is `NULL`. + row: sql + ? { note: null as unknown as undefined, ...mutation.modified } + : structuredClone(mutation.modified), + } + commits.push(await provider.write(request)) + if (mutant === `writes-insert-twice`) + commits.push(await provider.write(request)) + writeBack(mutation.modified) + } + return commits + }), + ), + onUpdate: ({ transaction }) => + answer( + persist(async () => { + const commits: Array = [] + for (const mutation of transaction.mutations) { + commits.push( + await provider.write({ + type: `update`, + id: mutation.key, + row: updateRow(mutation), + }), + ) + writeBack(mutation.modified) + } + return commits + }), + ), + onDelete: ({ transaction }) => + answer( + persist(async () => { + if (mutant === `skips-delete`) return [] + const commits: Array = [] + for (const mutation of transaction.mutations) { + commits.push( + await provider.write({ type: `delete`, id: mutation.key }), + ) + } + return commits + }), + ), + }) + return collection +} + +const sqlMutants = new Set([ + `sends-complete-row`, + `drops-removal`, + `settles-on-answer`, + `chains-on-confirmation`, + `confirms-by-high-water`, +]) + +/** A production driver for the reference adapter, optionally mutated. */ +export function referenceMutationDriver( + mutant?: ReferenceMutationMutant, + { + json = mutant === `echoes-authored-row`, + sql = sqlMutants.has(mutant), + log = false, + application = false, + inOrder = false, + confirms = sql, + }: ReferenceMutationOptions = {}, +): MutationDriver { + const capabilities: Array = [ + `write-rejection`, + `held-writes`, + `foreign-commits`, + `server-rewrites`, + ] + if (sql) capabilities.push(`field-updates`) + if (confirms) capabilities.push(`held-echoes`) + if (log) capabilities.push(`log-ordered-echoes`) + if (application) capabilities.push(`application-handlers`) + if (inOrder) capabilities.push(`ordered-answers`) + return { + adapter: `reference mutations${json ? ` (JSON provider)` : ``}${sql ? ` (SQL provider)` : ``}${log ? ` (log)` : ``}${application ? ` (application handlers)` : ``}${inOrder ? ` (in order)` : ``}${mutant ? ` (${mutant})` : ``}`, + ...(json && { undefinedField: `absent` as const }), + ...(sql && { undefinedField: `null` as const }), + capabilities: new Set(capabilities), + create(seed): MutationFixture { + const provider = new ControlledMutationProvider( + seed, + { json, sql, log, inOrder }, + mutant === `echoes-authored-row`, + ) + const collection = referenceCollection(provider, mutant, { + sql, + confirms, + application, + }) + return { + collection: observedView(collection, { + row: (row) => row, + key: (key) => key as number, + }), + base: () => collection.base.values(), + writer: { + insert: (row) => collection.insert(row), + update: (row) => + collection.update(row.id, (draft) => { + draft.title = row.title + if (Object.hasOwn(row, `note`)) draft.note = row.note + // Only a held note value is removed. + else if (typeof draft.note === `string`) delete draft.note + }), + delete: (id) => collection.delete(id), + }, + provider, + } + }, + } +} diff --git a/packages/db-adapter-conformance/tests/reference-mutation-oracle.property.test.ts b/packages/db-adapter-conformance/tests/reference-mutation-oracle.property.test.ts new file mode 100644 index 0000000000..7ae5496e9f --- /dev/null +++ b/packages/db-adapter-conformance/tests/reference-mutation-oracle.property.test.ts @@ -0,0 +1,185 @@ +import { fc } from '@fast-check/vitest' +import { describe, expect, it } from 'vitest' +import { + oraclePropertyOptions, + oracleReplayTest, + oracleRuns, +} from '../../db/tests/oracle-config' +import { declareOracle } from '../../db/tests/oracle-declaration' +import { createOracleReach } from '../../db/tests/oracle-reach' +import { + ConformanceViolation, + checkMutationHistory, + mutationHistoryArb, + mutationModel, + mutationProperty, + mutationReachStatesFor, +} from '../src' +import { referenceMutationDriver } from './reference-mutation-adapter' +import type { MutationDriver } from '../src' +import type { ReferenceMutationMutant } from './reference-mutation-adapter' + +/** + * # The mutation oracle on a conforming adapter + * + * The shared oracle in `src/mutation-oracle.ts` states the laws, the model, + * the history grammar, and the check. This file supplies one production + * driver: the reference adapter in `reference-mutation-adapter.ts`, which + * persists its own writes and declares every mutation capability. A green + * run here shows that the check accepts a conforming adapter; it says nothing + * about a published adapter, which needs its own driver. + * + * - **Campaigns.** The same property, grammar, and run budget run with a + * fixed seed and without one (ORC-007). Each fails when fewer than five + * histories reach a declared state (ORC-015). + * - **Value domain.** The fixed campaign also runs with a JSON provider, + * which declares `undefinedField: 'absent'`: a field an update removes is + * absent from its request and the provider's copy (FID-1's value domain). + * - **SQL provider.** The fixed campaign also runs with a provider like + * TrailBase's or PowerSync's: `undefinedField: 'null'`, `field-updates`, + * and `held-echoes`, so the adapter confirms each write through sync. + * - **Log provider, application handlers.** The fixed campaign also runs + * like Electric: a SQL provider whose change feed is a log + * (`log-ordered-echoes`) and handlers the application writes, which the + * adapter does not order (`application-handlers`). Another runs like + * Convex: a JSON provider that answers requests in arrival order + * (`ordered-answers`), with a log feed and application handlers. + * - **Calibration.** Each hostile mutant of the reference adapter breaks one + * law, and the fixed campaign must reject it with that law (ORC-006). + */ + +const property = `adapter-conformance.mutations.reference` +const driver = referenceMutationDriver() +const historyArb = mutationHistoryArb(driver) +const runs = 300 +const campaignTimeoutMs = 120_000 + +const jsonDriver = referenceMutationDriver(undefined, { json: true }) +const sqlDriver = referenceMutationDriver(undefined, { sql: true }) +const logDriver = referenceMutationDriver(undefined, { + sql: true, + log: true, + application: true, +}) +const inOrderDriver = referenceMutationDriver(undefined, { + json: true, + log: true, + application: true, + inOrder: true, + confirms: true, +}) + +async function campaign( + options: fc.Parameters<[unknown]>, + under: MutationDriver = driver, +) { + const reach = createOracleReach(property, mutationReachStatesFor(under), { + minimum: 5, + }) + await fc.assert(mutationProperty(under, reach), options) + return reach +} + +describe(`mutation conformance: reference adapter`, () => { + oracleReplayTest(it)( + `conforms on a fixed-seed campaign`, + async () => { + const reach = await campaign({ seed: 2118, numRuns: oracleRuns(runs) }) + reach.assertReached() + }, + campaignTimeoutMs, + ) + + oracleReplayTest(it, property)( + `conforms on a random campaign or a replayed seed`, + async () => { + const reach = await campaign(oraclePropertyOptions(runs, property)) + reach.assertReached() + }, + campaignTimeoutMs, + ) + + oracleReplayTest(it)( + `conforms with a JSON provider on a fixed-seed campaign`, + async () => { + const reach = await campaign( + { seed: 2118, numRuns: oracleRuns(runs) }, + jsonDriver, + ) + reach.assertReached() + }, + campaignTimeoutMs, + ) + + oracleReplayTest(it)( + `conforms with a SQL provider on a fixed-seed campaign`, + async () => { + const reach = await campaign( + { seed: 2118, numRuns: oracleRuns(runs) }, + sqlDriver, + ) + reach.assertReached() + }, + campaignTimeoutMs, + ) + + oracleReplayTest(it)( + `conforms with a SQL log provider and application handlers on a fixed-seed campaign`, + async () => { + const reach = await campaign( + { seed: 2118, numRuns: oracleRuns(runs) }, + logDriver, + ) + reach.assertReached() + }, + campaignTimeoutMs, + ) + + oracleReplayTest(it)( + `conforms with an in-order JSON log provider and application handlers on a fixed-seed campaign`, + async () => { + const reach = await campaign( + { seed: 2118, numRuns: oracleRuns(runs) }, + inOrderDriver, + ) + reach.assertReached() + }, + campaignTimeoutMs, + ) + + const mutants: Array<[ReferenceMutationMutant, string | RegExp]> = [ + [`applies-copy-late`, `MUT-1`], + [`swallows-rejection`, `MUT-2`], + [`wraps-rejection`, `MUT-2`], + [`writes-insert-twice`, `MUT-10/MUT-13`], + [`writes-concurrently`, `MUT-10/MUT-13`], + [`skips-delete`, /MUT-8|FID-5/], + [`sends-changes-only`, `MUT-8`], + [`echoes-authored-row`, /FID-5/], + [`writes-back-authored`, /FID-5/], + [`sends-complete-row`, `MUT-8 (provider requests)`], + [`drops-removal`, `MUT-8 (provider requests)`], + [`settles-on-answer`, /MUT-1|MUT-7/], + [`chains-on-confirmation`, /MUT-8|MUT-10/], + [`confirms-by-high-water`, /MUT-1|MUT-7/], + ] + oracleReplayTest(it).each(mutants)( + `rejects the %s mutant with %s`, + async (mutant, law) => { + const result = await fc.check( + mutationProperty(referenceMutationDriver(mutant)), + { seed: 2118, numRuns: runs }, + ) + expect(result.failed).toBe(true) + expect(result.errorInstance).toBeInstanceOf(ConformanceViolation) + expect((result.errorInstance as ConformanceViolation).law).toMatch(law) + }, + campaignTimeoutMs, + ) +}) + +declareOracle({ + check: [checkMutationHistory, mutationProperty], + model: [mutationModel.step, mutationModel.visible], + grammar: [historyArb], +}) diff --git a/packages/db/tests/collection-state-retention-oracle.property.test.ts b/packages/db/tests/collection-state-retention-oracle.property.test.ts index 2d94e82a4f..3da4555702 100644 --- a/packages/db/tests/collection-state-retention-oracle.property.test.ts +++ b/packages/db/tests/collection-state-retention-oracle.property.test.ts @@ -1202,21 +1202,36 @@ it.each( }) }, ) -fcTest.prop([optimisticHistory], { numRuns: oracleRuns(100), seed: 86103 })( - `matches optimistic ownership and publication histories with a fixed seed`, - async ({ initial, steps }) => { - await runOptimisticHistory(initial, steps) - }, -) -fcTest.prop( - [optimisticHistory], - oraclePropertyOptions(100, `collection-state.optimistic-history`), -)( - `matches optimistic ownership and publication histories with a random or replayed seed`, - async ({ initial, steps }) => { - await runOptimisticHistory(initial, steps) - }, -) +// Reach, per campaign (ORC-015): an edit that removes a field the row holds. +// The update API keeps the removed field as an own `undefined`, which only +// strict row comparisons distinguish from an absent one. +async function optimisticCampaign( + campaign: string, + options: Parameters[1], +): Promise { + const reach = createOracleReach( + `collection-state.optimistic-history (${campaign})`, + [`removed-held-field`], + { minimum: 5 }, + ) + await fc.assert( + fc.asyncProperty(optimisticHistory, async ({ initial, steps }) => { + reach.beginHistory() + const counts = await runOptimisticHistory(initial, steps) + if (counts.removedFieldEdits > 0) reach.reach(`removed-held-field`) + }), + options, + ) + reach.assertReached() +} + +it(`matches optimistic ownership and publication histories with a fixed seed`, () => + optimisticCampaign(`fixed`, { numRuns: oracleRuns(100), seed: 86103 })) +it(`matches optimistic ownership and publication histories with a random or replayed seed`, () => + optimisticCampaign( + `random`, + oraclePropertyOptions(100, `collection-state.optimistic-history`), + )) // An aborted open batch never applies, so the source's own rows must return // to their state before it. A later partial update then omits `c` again, and // the Collection keeps the held `c`. Shrunk from seed -526998278. diff --git a/packages/db/tests/optimistic-history-oracle.ts b/packages/db/tests/optimistic-history-oracle.ts index 90e21f885e..07b41016c7 100644 --- a/packages/db/tests/optimistic-history-oracle.ts +++ b/packages/db/tests/optimistic-history-oracle.ts @@ -140,7 +140,17 @@ import type { CollectionConfig, SyncConfig } from '../src/types.js' * fields. */ -export type HistoryRow = { id: number; a: number; b: number; c: number } +// `d` is optional: only an application's edit sets it, and an edit can +// remove it again. The update API represents a removed field as an own +// `undefined` field in the row and in `changes` (the documented rule in +// proxy-detachment-contract-oracle.test.ts; maintainer decision 2026-11-03). +export type HistoryRow = { + id: number + a: number + b: number + c: number + d?: number +} type Fields = Partial> type SourceOperation = { type: `row`; row: HistoryRow } | { type: `delete`; key: number } @@ -183,6 +193,8 @@ export type OptimisticStep = // Replace `fields` with the visible row's zero fields, signs flipped. // A missing row, or one with no zero, keeps `fields`. flipZeros?: boolean + // The draft also deletes `d` (`delete draft.d`). + removesD?: boolean } | { type: `delete` @@ -332,16 +344,20 @@ class HistoryModel { row && Object.entries(step.fields).every(([key, value]) => Object.is(row[key as keyof Fields], value), - ) + ) && + !(step.removesD && Object.hasOwn(row, `d`)) ) return + const authored: HistoryRow = { + ...(row ?? { id: step.key, a: 0, b: 0, c: this.insertDefault }), + ...(step.type === `edit` ? step.fields : {}), + } + if (step.type === `edit` && step.removesD && Object.hasOwn(authored, `d`)) + authored.d = undefined this.transactions.push({ key: step.key, kind: step.type === `delete` ? `delete` : row ? `update` : `insert`, - row: { - ...(row ?? { id: step.key, a: 0, b: 0, c: this.insertDefault }), - ...(step.type === `edit` ? step.fields : {}), - }, + row: authored, optimistic: step.optimistic, state: step.type === `edit` && step.pending ? `pending` : `persisting`, held: false, @@ -517,7 +533,14 @@ class HistoryModel { } } -const plain = ({ id, a, b, c }: HistoryRow): HistoryRow => ({ id, a, b, c }) +const plain = (row: HistoryRow): HistoryRow => ({ + id: row.id, + a: row.a, + b: row.b, + c: row.c, + // An own `d`, even `undefined`, stays; an absent one stays absent. + ...(Object.hasOwn(row, `d`) ? { d: row.d } : {}), +}) const observed = ( row: HistoryRow & { $origin: `local` | `remote` @@ -729,7 +752,8 @@ export const optimisticStep: fc.Arbitrary = fc.oneof( key: fc.integer({ min: 1, max: 3 }), fields: fc .record( - { a: fieldValue, b: fieldValue, c: fieldValue }, + // `d` too, so a later edit often finds a field to remove. + { a: fieldValue, b: fieldValue, c: fieldValue, d: fieldValue }, { requiredKeys: [] }, ) .filter((fields) => Object.keys(fields).length > 0), @@ -737,6 +761,29 @@ export const optimisticStep: fc.Arbitrary = fc.oneof( inHandler: handlerSourceBatch, }), }, + { + // An edit that sets the optional `d`, or removes it with `delete` + // (`removesD`). Removal needs a row that holds `d`, so both arms share + // a key range small enough to revisit. + weight: 2, + arbitrary: fc.oneof( + fc.record({ + type: fc.constant(`edit` as const), + key: fc.integer({ min: 1, max: 3 }), + fields: fieldValue.map((d): Fields => ({ d })), + optimistic: fc.boolean(), + inHandler: fc.constant(undefined), + }), + fc.record({ + type: fc.constant(`edit` as const), + key: fc.integer({ min: 1, max: 3 }), + fields: fc.constant({}), + optimistic: fc.boolean(), + inHandler: fc.constant(undefined), + removesD: fc.constant(true), + }), + ), + }, { // An edit that changes only the sign of the row's zeros (-0 and 0 are // distinct row values). Random fields rarely do it by chance. It @@ -785,7 +832,26 @@ export const optimisticHistory = fc.record({ selector: (row) => row.id, maxLength: 3, }), - steps: fc.array(optimisticStep, { minLength: 2, maxLength: 24 }), + steps: fc + .array(optimisticStep, { minLength: 2, maxLength: 24 }) + // A removal of `d` follows an edit that sets it on the same key, so the + // removal finds a field to remove; random steps rarely line them up. + .map((steps) => + steps.flatMap((step): Array => + step.type === `edit` && step.removesD + ? [ + { + type: `edit`, + key: step.key, + fields: { d: 1 }, + optimistic: step.optimistic, + inHandler: undefined, + }, + step, + ] + : [step], + ), + ), }) // The core grammar plus store steps. While a hold is outstanding, an // in-handler batch does not await acceptance. @@ -997,6 +1063,7 @@ export async function runOptimisticHistory( a: z.number(), b: z.number(), c: z.number().default(options.insertDefault), + d: z.number().optional(), }), }) const collection = @@ -1049,6 +1116,8 @@ export async function runOptimisticHistory( // visible row changed only by the sign of a zero: no pending-writes flag // changes, so only value equality can publish the change. zeroSignEditsOverPending: 0, + // Accepted edits that remove `d` from a row holding it. + removedFieldEdits: 0, // Drains that kept a settled row held for a deferred transaction. deferredHolds: 0, abortedDeferred: 0, @@ -1394,18 +1463,19 @@ export async function runOptimisticHistory( expect( actual, `${label}: reads ${JSON.stringify(actual)} expected ${JSON.stringify(expected)}`, - ).toEqual(expected) - expect(sorted(replica.values()), `${label}: event replica`).toEqual( - expected, - ) + ).toStrictEqual(expected) + expect( + sorted(replica.values()), + `${label}: event replica`, + ).toStrictEqual(expected) expect( sorted(rawReplica.values()), `${label}: raw event replica`, - ).toEqual(expected) + ).toStrictEqual(expected) expect( sorted([...downstream.values()].map(plain)), `${label}: downstream`, - ).toEqual(expected.map(plain)) + ).toStrictEqual(expected.map(plain)) // Every per-mutation pass walks the tracked transactions, so the // Collection tracks exactly its unsettled transactions. A held row // outlives its completed transaction in the held-row layer, not here. @@ -1451,6 +1521,14 @@ export async function runOptimisticHistory( if (index === undefined) continue const intent = model.transactions[index]! const was = before.find((row) => row.id === step.key) + // The edit removes a `d` the row holds (the documented own + // `undefined` in `changes`). + const removesHeldD = + step.type === `edit` && + step.removesD === true && + was !== undefined && + Object.hasOwn(was, `d`) + if (removesHeldD) counts.removedFieldEdits++ const now = model.visible().get(step.key) if ( inFlight && @@ -1499,7 +1577,10 @@ export async function runOptimisticHistory( : collection.update( step.key, { optimistic: step.optimistic }, - (draft) => Object.assign(draft, step.fields), + (draft) => { + Object.assign(draft, step.fields) + if (step.removesD) delete draft.d + }, ) const manual = step.type === `edit` && step.pending === true const tx = manual @@ -1517,7 +1598,9 @@ export async function runOptimisticHistory( expect( live && plain(live), `${position}: optimistic mutation is live in the same tick`, - ).toEqual(step.type === `delete` ? undefined : plain(intent.row)) + ).toStrictEqual( + step.type === `delete` ? undefined : plain(intent.row), + ) } const operation: (typeof operations)[number] = { tx, @@ -1544,9 +1627,10 @@ export async function runOptimisticHistory( : plain(intent.row) : // An authored field is a change unless it is the same // value; -0 and 0 are distinct row values (maintainer - // decision, 2026-10-29). - Object.fromEntries( - Object.entries(step.fields).filter( + // decision, 2026-10-29). A removed field is a change + // marked `undefined`. + Object.fromEntries([ + ...Object.entries(step.fields).filter( ([key, value]) => !Object.is( before.find((row) => row.id === intent.key)?.[ @@ -1555,7 +1639,8 @@ export async function runOptimisticHistory( value, ), ), - ), + ...(removesHeldD ? [[`d`, undefined]] : []), + ]), } operations.push(operation) expect( diff --git a/packages/db/tests/oracle-config.ts b/packages/db/tests/oracle-config.ts index 635bfa2149..ec0d7868bf 100644 --- a/packages/db/tests/oracle-config.ts +++ b/packages/db/tests/oracle-config.ts @@ -254,6 +254,12 @@ const staticOracleProperties = [ `sqlite-persistence.request-local-failure-buffered-source`, `sqlite-persistence.optimistic-history`, `adapter-conformance.eager-live-source.reference`, + `adapter-conformance.mutations.reference`, + `adapter-conformance.mutations.rxdb`, + `adapter-conformance.mutations.trailbase`, + `adapter-conformance.mutations.powersync`, + `adapter-conformance.mutations.electric`, + `adapter-conformance.mutations.convex`, `adapter-conformance.eager-live-source.neon-realtime`, `adapter-conformance.eager-live-source.trailbase`, `adapter-conformance.eager-live-source.rxdb`, diff --git a/packages/db/tests/oracle-replay-manifest.ts b/packages/db/tests/oracle-replay-manifest.ts index 927d86081d..f326954deb 100644 --- a/packages/db/tests/oracle-replay-manifest.ts +++ b/packages/db/tests/oracle-replay-manifest.ts @@ -818,6 +818,36 @@ const ownerGroups: ReadonlyArray< `adapter-conformance.eager-live-source`, `convex`, ], + [ + `db-adapter-conformance/tests/reference-mutation-oracle.property.test.ts`, + `adapter-conformance.mutations`, + `reference`, + ], + [ + `rxdb-db-collection/tests/mutation-conformance-oracle.property.test.ts`, + `adapter-conformance.mutations`, + `rxdb`, + ], + [ + `trailbase-db-collection/tests/mutation-conformance-oracle.property.test.ts`, + `adapter-conformance.mutations`, + `trailbase`, + ], + [ + `powersync-db-collection/tests/mutation-conformance-oracle.property.test.ts`, + `adapter-conformance.mutations`, + `powersync`, + ], + [ + `electric-db-collection/tests/mutation-conformance-oracle.property.test.ts`, + `adapter-conformance.mutations`, + `electric`, + ], + [ + `convex-db-collection/tests/mutation-conformance-oracle.property.test.ts`, + `adapter-conformance.mutations`, + `convex`, + ], ] const owners = new Map() @@ -919,6 +949,13 @@ const replayTestNames = new Map([ `conforms on a random campaign or a replayed seed`, ] as const, ), + ...[`reference`, `rxdb`, `trailbase`, `powersync`, `electric`, `convex`].map( + (driver) => + [ + `adapter-conformance.mutations.${driver}`, + `conforms on a random campaign or a replayed seed`, + ] as const, + ), [ `query-db.refetch-held-page-end`, `generated same-key page shifts.*replay campaign`, diff --git a/packages/electric-db-collection/tests/mutation-conformance-driver.ts b/packages/electric-db-collection/tests/mutation-conformance-driver.ts new file mode 100644 index 0000000000..9f46558bb4 --- /dev/null +++ b/packages/electric-db-collection/tests/mutation-conformance-driver.ts @@ -0,0 +1,386 @@ +/** + * Production driver for the shared mutation oracle + * (`packages/db-adapter-conformance/src/mutation-oracle.ts`). It runs the real + * Electric adapter in eager mode over the installed `@electric-sql/client` + * `ShapeStream`, which fetches through a controlled `fetchClient` that + * answers the Electric HTTP protocol, as `conformance-driver.ts` does. + * + * Electric does not persist writes: the application's handlers do, through + * its own backend API, and confirm each through the shape stream with the + * adapter's `awaitTxId`. The driver supplies such an application: + * + * - the provider is the backend and its Postgres table. Its API takes a + * write request, answers by plan, and on acceptance commits it in a new + * transaction and answers with that txid; + * - each handler sends the complete row as a SQL row (an absent `note` is + * `NULL`, `undefinedField: 'null'`), then awaits `awaitTxId(txid)`. The + * adapter does not order the handlers (`application-handlers`), so each + * write's request goes out when it is made; + * - each commit reaches the shape log as one batch: its change messages, + * carrying its txid, and `up-to-date`. An update carries the primary key + * and the changed columns, as in Electric's default replica mode. The log + * is in commit order (`log-ordered-echoes`); a held echo stalls it at that + * commit until `deliver` (`held-echoes`); + * - a rewrite (`server-rewrites`) is the backend storing a changed row, as a + * trigger or column default does; its message carries the stored row; + * - a commit by another client is a backend transaction that is not the + * application's. + * + * The application inserts an absent `note` as `null` and removes a held one + * by assigning `undefined`, the update API's removed field. Limits: one sync + * run; no reconnect or must-refetch while a write waits; `awaitMatch` is not + * exercised. + */ +import { createCollection } from '@tanstack/db' +import { electricCollectionOptions } from '../src/electric' +import { + acceptAtOnce, + fromSqlRow, + observedView, + rewriteRow, + settleWhenQuiet, + toSqlRow, +} from '../../db-adapter-conformance/src' +import type { + ConformanceRow, + MutationDriver, + MutationFixture, + MutationProvider, + ProviderChange, + SettleCut, + WritePlan, + WriteRequest, +} from '../../db-adapter-conformance/src' + +const SCHEMA = JSON.stringify({ + id: { type: `int4`, pk_index: 0 }, + title: { type: `text` }, + note: { type: `text` }, +}) + +type ElectricRow = { id: number; title: string | null; note: string | null } +type Wire = Record +type Batch = Array> + +/** A row as Electric sends it: every column as text, `NULL` as null. */ +function wire(row: ConformanceRow): Wire { + const sql = toSqlRow(row) + return { id: String(sql.id), title: sql.title, note: sql.note } +} + +type Request = { + url: URL + respond: (response: Response) => void +} + +class ElectricMutationProvider implements MutationProvider { + /** The backend's table. */ + private readonly table = new Map() + private readonly plans: Array = [] + private readonly held: Array<() => void> = [] + private readonly log: Array = [] + /** Committed batches the shape log has not sent, in commit order. */ + private readonly outbox: Array = [] + /** Batches the log holds at a held echo; `own` marks the holding one. */ + private readonly stalled: Array<{ batch: Batch; own: boolean }> = [] + private waiting: Request | undefined + private txid = 100 + private offset = 0 + /** Cleanup aborted the client's request: nothing more will arrive. */ + private closed = false + + constructor(seed: ReadonlyArray) { + for (const row of seed) this.table.set(row.id, row) + } + + // The shape stream's side. + + readonly fetchClient: typeof fetch = (input, init) => + new Promise((resolve, reject) => { + const url = new URL(String(input)) + const signal = init?.signal ?? undefined + const request: Request = { url, respond: resolve } + signal?.addEventListener( + `abort`, + () => { + if (this.waiting === request) this.waiting = undefined + this.closed = true + reject( + signal.reason ?? new DOMException(`request aborted`, `AbortError`), + ) + }, + { once: true }, + ) + if (url.searchParams.get(`offset`) === `-1`) { + this.respond(request, [ + ...[...this.table.values()].map((row) => ({ + key: String(row.id), + value: wire(row), + headers: { operation: `insert` }, + })), + ]) + return + } + this.waiting = request + this.flush() + }) + + private respond(request: Request, messages: Batch): void { + this.offset++ + request.respond( + new Response( + JSON.stringify([ + ...messages, + { + headers: { + control: `up-to-date`, + global_last_seen_lsn: String(this.offset), + }, + }, + ]), + { + headers: { + 'electric-handle': `mutations`, + 'electric-offset': `${this.offset}_0`, + 'electric-cursor': String(this.offset), + 'electric-schema': SCHEMA, + 'electric-up-to-date': `true`, + }, + }, + ), + ) + } + + /** Sends the next unsent batch to the client's waiting live request. */ + private flush(): void { + const request = this.waiting + if (!request || this.outbox.length === 0) return + this.waiting = undefined + this.respond(request, this.outbox.shift()!) + } + + /** Commits a change in a new transaction and queues its batch. */ + private commit(change: ProviderChange, holdEcho: boolean): number { + const txid = ++this.txid + let message: Record | undefined + if (change.type === `delete`) { + if (this.table.delete(change.id)) + message = { + key: String(change.id), + value: { id: String(change.id) }, + headers: { operation: `delete`, txids: [txid] }, + } + } else { + const held = this.table.get(change.row.id) + this.table.set(change.row.id, change.row) + const after = wire(change.row) + const before = held && wire(held) + message = { + key: String(change.row.id), + // Default replica mode: the key and the changed columns. + value: before + ? Object.fromEntries( + Object.entries(after).filter( + ([name, value]) => name === `id` || before[name] !== value, + ), + ) + : after, + headers: { operation: before ? `update` : `insert`, txids: [txid] }, + } + } + const batch = message ? [message] : [] + // The log is in commit order: a batch waits behind a held echo. + if (holdEcho || this.stalled.length > 0) + this.stalled.push({ batch, own: holdEcho }) + else this.outbox.push(batch) + this.flush() + return txid + } + + // The application's side: its backend API. + + /** One write request; answers with the commit's txid. */ + api(request: WriteRequest): Promise<{ txid: number }> { + this.log.push(JSON.parse(JSON.stringify(request)) as WriteRequest) + const plan = this.plans.shift() ?? acceptAtOnce + return new Promise((resolve, reject) => { + const answer = () => { + if (plan.outcome === `reject`) { + reject(plan.error) + return + } + const change: ProviderChange = + request.type === `delete` + ? { type: `delete`, id: request.id } + : { + type: `upsert`, + row: rewriteRow(fromSqlRow({ ...request.row }), plan.rewrite), + } + resolve({ txid: this.commit(change, plan.echoHeld) }) + } + if (plan.held) this.held.push(answer) + else answer() + }) + } + + // The oracle's side. + + plan(plan: WritePlan): void { + this.plans.push(plan) + } + + release(): void { + this.held.shift()?.() + } + + /** The log resumes at the oldest held echo, up to the next one. */ + deliver(): void { + const first = this.stalled.shift() + if (!first) return + this.outbox.push(first.batch) + while (this.stalled[0] && !this.stalled[0].own) + this.outbox.push(this.stalled.shift()!.batch) + this.flush() + } + + /** Answers every held request and resumes the log. */ + drain(): void { + for (const answer of this.held.splice(0)) answer() + while (this.stalled.length > 0) this.deliver() + } + + foreignCommit(change: ProviderChange): void { + this.commit(change, false) + } + + rows(): ReadonlyArray { + return [...this.table.values()] + } + + requests(): ReadonlyArray { + return this.log + } + + /** + * The propagation step: every batch sent and the client asking again, + * and every write's receipt settled; while a request or an echo is held, + * until the request log, the sent batches, and the receipts are quiet. + */ + settle = ({ unsettled }: SettleCut) => { + const delivered = () => + this.outbox.length === 0 && this.waiting !== undefined + return settleWhenQuiet({ + done: () => delivered() && unsettled() === 0, + blocked: () => + delivered() && (this.held.length > 0 || this.stalled.length > 0), + activity: () => + `${this.log.length} ${this.offset} ${unsettled()} ${this.outbox.length}`, + stopped: () => this.closed, + }) + } +} + +let sequence = 0 + +export function electricMutationDriver(): MutationDriver { + return { + adapter: `@tanstack/electric-db-collection (eager)`, + capabilities: new Set([ + `write-rejection`, + `held-writes`, + `foreign-commits`, + `server-rewrites`, + `held-echoes`, + `log-ordered-echoes`, + `application-handlers`, + ]), + undefinedField: `null`, + create(seed): MutationFixture { + const provider = new ElectricMutationProvider(seed) + // The application's handlers: write through its API, then wait for the + // write's transaction in the shape stream. + const persist = async ( + collection: { + utils: { awaitTxId: (txid: number) => Promise } + }, + requests: Array, + ) => { + for (const request of requests) { + const { txid } = await provider.api(request) + await collection.utils.awaitTxId(txid) + } + } + const collection = createCollection( + electricCollectionOptions({ + id: `electric-mutations-${++sequence}`, + shapeOptions: { + url: `http://conformance/mutations-${sequence}`, + params: { table: `rows` }, + fetchClient: provider.fetchClient, + }, + getKey: (row) => row.id, + onInsert: ({ transaction, collection: target }) => + persist( + target, + transaction.mutations.map((mutation) => ({ + type: `insert`, + id: mutation.modified.id, + // A SQL row has every column: an absent `note` is `NULL`. + row: toSqlRow( + fromSqlRow(mutation.modified), + ) as Partial, + })), + ), + onUpdate: ({ transaction, collection: target }) => + persist( + target, + transaction.mutations.map((mutation) => ({ + type: `update`, + id: mutation.modified.id, + // The complete row: a removed field is `NULL`. + row: { + id: mutation.modified.id, + title: mutation.modified.title, + note: mutation.modified.note ?? null, + } as Partial, + })), + ), + onDelete: ({ transaction, collection: target }) => + persist( + target, + transaction.mutations.map((mutation) => ({ + type: `delete`, + id: mutation.original.id, + })), + ), + }), + ) + return { + collection: observedView(collection, { + row: fromSqlRow, + key: (key) => key as number, + }), + base: () => [...collection.base.values()].map(fromSqlRow), + writer: { + insert: (row) => { + const { id, title, note } = toSqlRow(row) + return collection.insert({ id, title, note }) + }, + update: (row) => + collection.update(row.id, (draft) => { + draft.title = row.title + if (Object.hasOwn(row, `note`)) draft.note = row.note! + // A `NULL` note is already absent; only a held one is removed. + else if (typeof draft.note === `string`) + (draft as { note?: string | null }).note = undefined + }), + delete: (id) => collection.delete(id), + }, + provider, + settle: provider.settle, + drain: () => provider.drain(), + dispose: () => collection.cleanup(), + } + }, + } +} diff --git a/packages/electric-db-collection/tests/mutation-conformance-oracle.property.test.ts b/packages/electric-db-collection/tests/mutation-conformance-oracle.property.test.ts new file mode 100644 index 0000000000..0c69936188 --- /dev/null +++ b/packages/electric-db-collection/tests/mutation-conformance-oracle.property.test.ts @@ -0,0 +1,77 @@ +import { fc } from '@fast-check/vitest' +import { describe, it } from 'vitest' +import { + oraclePropertyOptions, + oracleReplayTest, + oracleRuns, +} from '../../db/tests/oracle-config' +import { declareOracle } from '../../db/tests/oracle-declaration' +import { createOracleReach } from '../../db/tests/oracle-reach' +import { + checkMutationHistory, + mutationHistoryArb, + mutationModel, + mutationProperty, + mutationReachStatesFor, +} from '../../db-adapter-conformance/src' +import { electricMutationDriver } from './mutation-conformance-driver' + +/** + * # Electric on the shared mutation oracle + * + * A receiving driver for `packages/db-adapter-conformance`. The shared + * module states the laws (MUT-1, MUT-2, MUT-7, MUT-8, MUT-10, MUT-13, FID-5, + * HEALTH-5), the model, the history grammar, and the check. This file + * supplies only the Electric production driver in + * `mutation-conformance-driver.ts` and the campaigns; it states no expected + * results. + * + * Electric does not persist writes: the driver's application handlers write + * through a backend API and confirm each with the adapter's `awaitTxId`, so + * this checks what Electric owns, the confirmation through its shape stream + * (MUT-7), with the handlers' requests and the backend's rows as the + * application's. The backend can reject or hold a write, rewrite a stored + * row, and stall its shape log at a commit (`held-echoes`, + * `log-ordered-echoes`); another client can write; and the adapter does not + * order the application's handlers (`application-handlers`). + */ + +const property = `adapter-conformance.mutations.electric` +const driver = electricMutationDriver() +const historyArb = mutationHistoryArb(driver) +const runs = 300 +const campaignTimeoutMs = 120_000 + +async function campaign(options: fc.Parameters<[unknown]>) { + const reach = createOracleReach(property, mutationReachStatesFor(driver), { + minimum: 5, + }) + await fc.assert(mutationProperty(driver, reach), options) + return reach +} + +describe(`mutation conformance: Electric`, () => { + oracleReplayTest(it)( + `conforms on a fixed-seed campaign`, + async () => { + const reach = await campaign({ seed: 2118, numRuns: oracleRuns(runs) }) + reach.assertReached() + }, + campaignTimeoutMs, + ) + + oracleReplayTest(it, property)( + `conforms on a random campaign or a replayed seed`, + async () => { + const reach = await campaign(oraclePropertyOptions(runs, property)) + reach.assertReached() + }, + campaignTimeoutMs, + ) +}) + +declareOracle({ + check: [checkMutationHistory, mutationProperty], + model: [mutationModel.step, mutationModel.visible], + grammar: [historyArb], +}) diff --git a/packages/powersync-db-collection/tests/mutation-conformance-driver.ts b/packages/powersync-db-collection/tests/mutation-conformance-driver.ts new file mode 100644 index 0000000000..67fb60a199 --- /dev/null +++ b/packages/powersync-db-collection/tests/mutation-conformance-driver.ts @@ -0,0 +1,347 @@ +/** + * Production driver for the shared mutation oracle + * (`packages/db-adapter-conformance/src/mutation-oracle.ts`). It runs the real + * PowerSync adapter in eager mode over a real local PowerSync database + * (SQLite through `@powersync/node`), as `conformance-driver.ts` does. The + * adapter persists a write to that database and settles once its own diff + * trigger has observed it, so the provider is the local database; uploading + * to a PowerSync service is the SDK's upload queue, outside this oracle. + * + * The oracle's steps map onto PowerSync like this: + * + * - a write request is one `INSERT`, `UPDATE`, or `DELETE` statement the + * adapter executes on the table inside its write transaction. An insert + * sends every column; an update sets only its changed columns + * (`field-updates`); the request carries the bound values; + * - a plan answers the next request inside the adapter's transaction: a + * rejection throws the plan's error from the statement, as SQLite does for + * a failed write, which rolls the transaction back. The driver does not + * declare `held-writes`: a held statement holds the database's one write + * lock, which the adapter's diff flush also needs, so it would delay the + * confirmation of earlier writes, a coupling the model does not represent; + * - the adapter applies the database's changes through its change watcher + * (`onChangeWithCallback`), which runs at most every 30 ms, so each write's + * copy arrives after the database answers; the adapter must not settle + * the write before it applies (MUT-7), which the publication check sees. + * The driver does not declare `held-echoes`: one watcher callback applies + * every committed write, so it cannot withhold one write's copy while a + * later write proceeds; + * - a commit by another client is a write to the table that is not the + * adapter's, as the SDK's download of server changes is; + * - the provider's rows are the table's rows. `note` is a nullable text + * column, so an absent `note` is SQL `NULL` (`undefinedField: 'null'`, + * `toSqlRow`, `fromSqlRow`). + * + * PowerSync keys are strings, so row `n` is id `"n"`. The application + * inserts an absent `note` as `null` and removes a held one by assigning + * `undefined`, the update API's removed field; removal by assigning `null` + * is not generated. The local database does not rewrite rows, so the driver + * does not declare `server-rewrites`. + */ +import { randomUUID } from 'node:crypto' +import { rm } from 'node:fs/promises' +import { tmpdir } from 'node:os' +import { join } from 'node:path' +import { PowerSyncDatabase, Schema, Table, column } from '@powersync/node' +import { createCollection } from '@tanstack/db' +import { powerSyncCollectionOptions } from '../src' +import { + acceptAtOnce, + conformanceKey, + fromSqlRow, + observedView, + toSqlRow, +} from '../../db-adapter-conformance/src' +import { TEST_DATABASE_IMPLEMENTATION } from './test-db-implementation' +import type { LockContext } from '@powersync/node' +import type { + ConformanceRow, + MutationDriver, + MutationFixture, + MutationProvider, + ProviderChange, + SettleCut, + WritePlan, + WriteRequest, +} from '../../db-adapter-conformance/src' + +const SCHEMA = new Schema({ + rows: new Table({ + title: column.text, + note: column.text, + }), +}) + +type Record = { id: string; title: string | null; note: string | null } + +const sleep = (ms: number) => new Promise((resolve) => setTimeout(resolve, ms)) +const drainTimeoutMs = 5_000 + +/** Field names in a statement, without the quoting `sanitizeSQL` adds. */ +const unquote = (name: string) => name.trim().replace(/^['"]|['"]$/g, ``) +const names = (list: string) => list.split(`,`).map(unquote) + +/** + * The request an adapter statement makes, or undefined for a statement that + * is not a write to the table. + */ +function writeRequest( + sql: string, + params: ReadonlyArray, +): WriteRequest | undefined { + const insert = /^\s*INSERT\s+into\s+['"]?rows['"]?\s*\(([^)]*)\)/i.exec(sql) + if (insert) { + const row = Object.fromEntries( + names(insert[1]!).map((name, index) => [name, params[index]]), + ) as Record + return { + type: `insert`, + id: conformanceKey(row.id), + row: { ...row, id: conformanceKey(row.id) } as Partial, + } + } + const update = + /^\s*UPDATE\s+['"]?rows['"]?\s+SET\s+([\s\S]*?)\s+WHERE\s+id\s*=\s*\?/i.exec( + sql, + ) + if (update) { + const fields = names(update[1]!).map((set) => set.split(`=`)[0]!.trim()) + const id = conformanceKey(String(params.at(-1))) + return { + type: `update`, + id, + row: { + id, + ...Object.fromEntries( + fields.map((field, index) => [unquote(field), params[index]]), + ), + } as Partial, + } + } + const remove = + /^\s*DELETE\s+FROM\s+['"]?rows['"]?\s+WHERE\s+id\s*=\s*\?/i.exec(sql) + if (remove) return { type: `delete`, id: conformanceKey(String(params[0])) } + return undefined +} + +class PowerSyncMutationProvider implements MutationProvider { + readonly database: PowerSyncDatabase + readonly file: string + private readonly plans: Array = [] + private readonly log: Array = [] + private running = 0 + private drained = 0 + private drainTarget: number | undefined + private readonly rawWriteTransaction: PowerSyncDatabase[`writeTransaction`] + + constructor() { + if (!TEST_DATABASE_IMPLEMENTATION) { + throw new Error(`no SQLite implementation is available`) + } + this.file = `mutations-${randomUUID()}.sqlite` + this.database = new PowerSyncDatabase({ + database: { + dbFilename: this.file, + dbLocation: tmpdir(), + implementation: TEST_DATABASE_IMPLEMENTATION, + }, + schema: SCHEMA, + }) + this.rawWriteTransaction = this.database.writeTransaction.bind( + this.database, + ) + // The adapter's write transactions: each table write is a request. + this.database.writeTransaction = (async ( + callback: (tx: LockContext) => Promise, + lockTimeout?: number, + ) => { + // Set by the statement wrapper inside the transaction callback. + let wrote = false as boolean + const result = await this.rawWriteTransaction(async (tx) => { + const execute = tx.execute.bind(tx) + const guarded = Object.create(tx) as LockContext + guarded.execute = async (sql: string, params: Array = []) => { + const request = writeRequest(sql, params) + if (!request) return execute(sql, params) + this.log.push(request) + const plan = this.plans.shift() ?? acceptAtOnce + if (plan.outcome === `reject`) throw plan.error + wrote = true + return execute(sql, params) + } + return callback(guarded) + }, lockTimeout) + if (wrote) this.expectDrain() + return result + }) as PowerSyncDatabase[`writeTransaction`] + // The adapter's change watcher: count its drains. + const register = this.database.onChangeWithCallback.bind(this.database) + this.database.onChangeWithCallback = (handler, options) => + register( + { + ...handler, + onChange: async (event) => { + this.running++ + try { + await handler?.onChange(event) + } finally { + this.running-- + this.drained++ + } + }, + }, + options, + ) + } + + /** + * A commit is applied by a watcher callback that starts after it: one + * already running read the tracking table before the commit. + */ + private expectDrain(): void { + this.drainTarget = this.drained + this.running + 1 + } + + /** + * Writes that are not the adapter's: the seed and other clients. A table + * is a view, so an upsert is an update or an insert. A delete of a row the + * table lacks changes nothing, so the watcher has nothing to drain. + */ + private async write(change: ProviderChange): Promise { + const changed = await this.rawWriteTransaction(async (tx) => { + // A view's `INSTEAD OF` trigger reports no affected rows, so the row's + // presence decides whether the write changes anything. + if (change.type === `delete`) { + const id = String(change.id) + const held = await tx.getOptional(`SELECT id FROM rows WHERE id = ?`, [ + id, + ]) + if (held) await tx.execute(`DELETE FROM rows WHERE id = ?`, [id]) + return held !== null + } + const { id, title, note } = toSqlRow(change.row) + const held = await tx.getOptional(`SELECT id FROM rows WHERE id = ?`, [ + String(id), + ]) + if (held) + await tx.execute(`UPDATE rows SET title = ?, note = ? WHERE id = ?`, [ + title, + note, + String(id), + ]) + else + await tx.execute( + `INSERT INTO rows (id, title, note) VALUES (?, ?, ?)`, + [String(id), title, note], + ) + return true + }) + if (changed) this.expectDrain() + } + + async seed(rows: ReadonlyArray): Promise { + for (const row of rows) await this.write({ type: `upsert`, row }) + this.drainTarget = undefined + } + + plan(plan: WritePlan): void { + this.plans.push(plan) + } + + foreignCommit(change: ProviderChange): Promise { + return this.write(change) + } + + async rows(): Promise> { + const records = await this.database.getAll( + `SELECT id, title, note FROM rows ORDER BY id`, + ) + return records.map(conformanceRow) + } + + requests(): ReadonlyArray { + return this.log + } + + /** + * The propagation step (`propagationStep`: the diff-trigger watch). The + * adapter reaches its write transaction a few milliseconds after a write, + * and the watcher runs at most every 30 ms, so the step waits until every + * write's receipt has settled and the watcher has drained every commit. + */ + settle = async ({ unsettled }: SettleCut) => { + const deadline = Date.now() + drainTimeoutMs + while (Date.now() < deadline) { + const drainedAll = + (this.drainTarget === undefined || this.drained >= this.drainTarget) && + this.running === 0 + if (unsettled() === 0 && drainedAll) { + this.drainTarget = undefined + return + } + await sleep(5) + } + } + + dispose = async () => { + await this.database.disconnectAndClear() + await this.database.close() + await rm(join(tmpdir(), this.file), { force: true }) + } +} + +/** A table row as a conformance row: key `"n"` is row id `n`. */ +function conformanceRow(record: Partial): ConformanceRow { + return fromSqlRow({ ...record, id: conformanceKey(String(record.id)) }) +} + +export function powerSyncMutationDriver(): MutationDriver { + return { + adapter: `@tanstack/powersync-db-collection (eager)`, + capabilities: new Set([ + `write-rejection`, + `foreign-commits`, + `field-updates`, + ]), + undefinedField: `null`, + async create(seed): Promise { + const provider = new PowerSyncMutationProvider() + await provider.seed(seed) + const collection = createCollection( + powerSyncCollectionOptions({ + database: provider.database, + table: SCHEMA.props.rows, + }), + ) + const view = observedView(collection, { + row: conformanceRow, + key: conformanceKey, + }) + return { + collection: view, + base: () => [...collection.base.values()].map(conformanceRow), + writer: { + insert: (row) => { + const { id, title, note } = toSqlRow(row) + return collection.insert({ id: String(id), title, note }) + }, + update: (row) => + collection.update(String(row.id), (draft) => { + draft.title = row.title + if (Object.hasOwn(row, `note`)) draft.note = row.note! + // A `NULL` note is already absent; only a held one is removed. + else if (typeof draft.note === `string`) + (draft as { note?: string | null }).note = undefined + }), + delete: (id) => collection.delete(String(id)), + }, + provider, + settle: provider.settle, + dispose: async () => { + await collection.cleanup() + await provider.dispose() + }, + } + }, + } +} diff --git a/packages/powersync-db-collection/tests/mutation-conformance-oracle.property.test.ts b/packages/powersync-db-collection/tests/mutation-conformance-oracle.property.test.ts new file mode 100644 index 0000000000..7ba874d6cb --- /dev/null +++ b/packages/powersync-db-collection/tests/mutation-conformance-oracle.property.test.ts @@ -0,0 +1,76 @@ +import { fc } from '@fast-check/vitest' +import { describe, it } from 'vitest' +import { + oraclePropertyOptions, + oracleReplayTest, + oracleRuns, +} from '../../db/tests/oracle-config' +import { declareOracle } from '../../db/tests/oracle-declaration' +import { createOracleReach } from '../../db/tests/oracle-reach' +import { + checkMutationHistory, + mutationHistoryArb, + mutationModel, + mutationProperty, + mutationReachStatesFor, +} from '../../db-adapter-conformance/src' +import { powerSyncMutationDriver } from './mutation-conformance-driver' + +/** + * # PowerSync on the shared mutation oracle + * + * A receiving driver for `packages/db-adapter-conformance`. The shared + * module states the laws (MUT-1, MUT-2, MUT-7, MUT-8, MUT-10, MUT-13, FID-5, + * HEALTH-5), the model, the history grammar, and the check. This file + * supplies only the PowerSync production driver in + * `mutation-conformance-driver.ts` and the campaigns; it states no expected + * results. + * + * The driver runs a real local PowerSync database, the adapter's provider: + * the adapter persists each write to it and settles once its diff trigger + * has observed the write (`confirm-through-sync`, MUT-7). The database can + * reject or hold a write, another client can write, and updates set only + * their changed columns (the law set's `field-updates`). + */ + +const property = `adapter-conformance.mutations.powersync` +const driver = powerSyncMutationDriver() +const historyArb = mutationHistoryArb(driver) +const runs = 300 +// Each history opens, clears, and closes its own SQLite database, so a +// campaign takes about 47 s locally; the budget leaves room for CI. +const campaignTimeoutMs = 240_000 + +async function campaign(options: fc.Parameters<[unknown]>) { + const reach = createOracleReach(property, mutationReachStatesFor(driver), { + minimum: 5, + }) + await fc.assert(mutationProperty(driver, reach), options) + return reach +} + +describe(`mutation conformance: PowerSync`, () => { + oracleReplayTest(it)( + `conforms on a fixed-seed campaign`, + async () => { + const reach = await campaign({ seed: 2118, numRuns: oracleRuns(runs) }) + reach.assertReached() + }, + campaignTimeoutMs, + ) + + oracleReplayTest(it, property)( + `conforms on a random campaign or a replayed seed`, + async () => { + const reach = await campaign(oraclePropertyOptions(runs, property)) + reach.assertReached() + }, + campaignTimeoutMs, + ) +}) + +declareOracle({ + check: [checkMutationHistory, mutationProperty], + model: [mutationModel.step, mutationModel.visible], + grammar: [historyArb], +}) diff --git a/packages/rxdb-db-collection/package.json b/packages/rxdb-db-collection/package.json index c1d00f2275..9e6803c4bc 100644 --- a/packages/rxdb-db-collection/package.json +++ b/packages/rxdb-db-collection/package.json @@ -66,10 +66,10 @@ "@tanstack/db": "workspace:*", "@tanstack/store": "^0.9.2", "debug": "^4.4.3", - "rxdb": "17.1.0" + "rxdb": "17.6.0" }, "peerDependencies": { - "rxdb": ">=17.0.0", + "rxdb": ">=17.6.0", "rxjs": ">=7.8.2", "typescript": ">=4.7" }, diff --git a/packages/rxdb-db-collection/src/rxdb.ts b/packages/rxdb-db-collection/src/rxdb.ts index e9d36b1fe1..84c16ca641 100644 --- a/packages/rxdb-db-collection/src/rxdb.ts +++ b/packages/rxdb-db-collection/src/rxdb.ts @@ -7,12 +7,16 @@ import { rxStorageWriteErrorToRxError, } from 'rxdb/plugins/core' import DebugModule from 'debug' -import { withCollectionConfigFactory } from '@tanstack/db' +import { + UpdateKeyNotFoundError, + withCollectionConfigFactory, +} from '@tanstack/db' import { stripRxdbFields } from './helper' import type { FilledMangoQuery, RxCollection, RxDocumentData, + RxStorageWriteError, } from 'rxdb/plugins/core' import type { Subscription } from 'rxjs' @@ -300,47 +304,60 @@ export function rxdbCollectionOptions( rowUpdateMode: `full`, } + // Writes reach RxDB in author order: each starts after the earlier one is + // decided, whether it fulfilled or rejected. + let turn: Promise = Promise.resolve() + const inTurn = (write: () => Promise): Promise => { + const run = turn.then(write) + turn = run.catch(() => {}) + return run + } + const throwWriteError = (result: { + error: Array> + }) => { + if (result.error.length > 0) { + throw rxStorageWriteErrorToRxError(ensureNotFalsy(result.error[0])) + } + } + const collectionConfig: CollectionConfig = { ...restConfig, getKey, sync, - onInsert: async (params) => { - debug(`insert`, params) - const newItems = params.transaction.mutations.map((m) => m.modified) - const result = await rxCollection.bulkUpsert(newItems) - if (result.error.length > 0) { - throw rxStorageWriteErrorToRxError(ensureNotFalsy(result.error[0])) - } - }, - onUpdate: async (params) => { - debug(`update`, params) - const mutations = params.transaction.mutations.filter( - // eslint-disable-next-line @typescript-eslint/no-unnecessary-condition - (m) => m.type === `update`, - ) + onInsert: (params) => + inTurn(async () => { + debug(`insert`, params) + const newItems = params.transaction.mutations.map((m) => m.modified) + throwWriteError(await rxCollection.bulkUpsert(newItems)) + }), + onUpdate: (params) => + inTurn(async () => { + debug(`update`, params) + const mutations = params.transaction.mutations.filter( + // eslint-disable-next-line @typescript-eslint/no-unnecessary-condition + (m) => m.type === `update`, + ) - for (const mutation of mutations) { - const newValue = stripRxdbFields(mutation.modified) - const id = getKey(newValue) - const doc = await rxCollection.findOne(id).exec() - if (!doc) { - continue + for (const mutation of mutations) { + const newValue = stripRxdbFields(mutation.modified) + const id = getKey(newValue) + const doc = await rxCollection.findOne(id).exec() + // Another client deleted it while the update waited: the update + // cannot persist, so it fails rather than report success. + if (!doc) throw new UpdateKeyNotFoundError(id) + await doc.incrementalPatch(newValue) } - await doc.incrementalPatch(newValue) - } - }, - onDelete: async (params) => { - debug(`delete`, params) - const mutations = params.transaction.mutations.filter( - // eslint-disable-next-line @typescript-eslint/no-unnecessary-condition - (m) => m.type === `delete`, - ) - const ids = mutations.map((mutation) => getKey(mutation.original)) - const result = await rxCollection.bulkRemove(ids) - if (result.error.length > 0) { - throw result.error - } - }, + }), + onDelete: (params) => + inTurn(async () => { + debug(`delete`, params) + const mutations = params.transaction.mutations.filter( + // eslint-disable-next-line @typescript-eslint/no-unnecessary-condition + (m) => m.type === `delete`, + ) + const ids = mutations.map((mutation) => getKey(mutation.original)) + throwWriteError(await rxCollection.bulkRemove(ids)) + }), } return withCollectionConfigFactory(collectionConfig, () => rxdbCollectionOptions(config), diff --git a/packages/rxdb-db-collection/tests/mutation-conformance-driver.ts b/packages/rxdb-db-collection/tests/mutation-conformance-driver.ts new file mode 100644 index 0000000000..9555ccf698 --- /dev/null +++ b/packages/rxdb-db-collection/tests/mutation-conformance-driver.ts @@ -0,0 +1,395 @@ +/** + * Production driver for the shared mutation oracle + * (`packages/db-adapter-conformance/src/mutation-oracle.ts`). It runs the real + * RxDB adapter on a real RxDB database over RxDB's memory storage. The + * provider is a wrapper around that storage's `bulkWrite`, the one call + * through which every RxDB write reaches storage, so the adapter's own + * `bulkUpsert`, `findOne`, `incrementalPatch`, and `bulkRemove` calls run as + * an application's would. + * + * The oracle's steps map onto RxDB like this: + * + * - the provider stores each document as JSON, as RxDB's persistent + * storages and replication do, so a field whose value is `undefined` is + * absent from it (`undefinedField: 'absent'`); + * - a write request is one row of a `bulkWrite` call: an insert when the row + * has no live previous document, a delete when its document is + * `_deleted`, and an update otherwise. The request carries the document's + * fields without RxDB's reserved fields. A row the storage refuses as a + * conflict, an insert of a document it holds, is not a request: it has no + * effect, and RxDB's `bulkUpsert` sends it and answers it itself before + * it writes the update; + * - a plan answers the next request. A rejection writes nothing, in one of + * two kinds: `write-error` answers with a per-row write error in the + * `bulkWrite` response, as RxDB's schema validation refuses a document, + * which RxDB reports as an `RxError` whose `parameters.writeError` is that + * error object; `thrown` throws the plan's error from `bulkWrite`, as a + * failing storage does, which reaches the caller as itself + * (`matchesFailure`). + * A held request waits in the wrapper until `release`. An accepted + * request runs the memory storage's own `bulkWrite`; + * - a commit from another client is a write through the same RxCollection + * (`upsert`, or `remove` of the found document), as a second tab or a + * replication writes into the database. It bypasses the plans and the + * request log; + * - a rewrite (`server-rewrites`) is RxDB's `preInsert` and `preSave` hooks, + * which change a document before RxDB writes it, as an application's + * hooks for defaults or normalization do. The request keeps the document + * as the adapter sent it; + * - the provider's rows are the storage's non-deleted documents, read from + * the storage instance rather than RxDB's document cache. + * + * RxDB primary keys are strings. The driver stores row `n` as document id + * `"n"`, and `observedView` maps each key and row id back with + * `conformanceKey`. A JSON document omits an absent `note`. + * + * Limits: a conflict (status 409) is not generated. + */ +import { createCollection } from '@tanstack/db' +import { createRxDatabase, prepareQuery } from 'rxdb/plugins/core' +import { getRxStorageMemory } from 'rxdb/plugins/storage-memory' +import { rxdbCollectionOptions } from '../src/rxdb' +import { + acceptAtOnce, + conformanceKey, + observedView, + rewriteRow, +} from '../../db-adapter-conformance/src' +import type { + BulkWriteRow, + RxCollection, + RxDatabase, + RxDocumentData, + RxStorageBulkWriteResponse, + RxStorageInstance, + RxStorageWriteError, +} from 'rxdb/plugins/core' +import type { + ConformanceRow, + MutationDriver, + MutationFixture, + MutationProvider, + ProviderChange, + WritePlan, + WriteRequest, +} from '../../db-adapter-conformance/src' + +type Doc = { id: string; title: string; note?: string } + +const schema = { + version: 0, + type: `object`, + primaryKey: `id`, + properties: { + id: { type: `string`, maxLength: 10 }, + title: { type: `string` }, + note: { type: `string` }, + }, + required: [`id`, `title`], +} as const + +/** + * An RxDB document as a conformance row: key `"n"` is row id `n`. Other + * fields pass through, so an optimistic row's own `undefined` stays, and + * RxDB's reserved fields go. + */ +function conformanceRow(doc: Doc): ConformanceRow { + const { _rev, _meta, _deleted, _attachments, ...fields } = + doc as RxDocumentData + return { ...fields, id: conformanceKey(doc.id) } +} + +function document(row: ConformanceRow): Doc { + return { + id: String(row.id), + title: row.title, + ...(row.note === undefined ? {} : { note: row.note }), + } +} + +/** + * The request a write row makes. Its row is the document as the adapter sent + * it, before RxDB's hooks rewrote it (`sent`), stored as JSON. + */ +function writeRequest(row: BulkWriteRow, sent?: Doc): WriteRequest { + const id = conformanceKey(row.document.id) + if (row.document._deleted) return { type: `delete`, id } + return { + type: row.previous && !row.previous._deleted ? `update` : `insert`, + id, + row: conformanceRow( + sent ? (JSON.parse(JSON.stringify(sent)) as Doc) : row.document, + ), + } +} + +/** + * A write row as a serializing storage stores it. RxDB documents are JSON, + * and its persistent storages and replication serialize them, so the + * provider stores JSON: a field whose value is `undefined` is absent. The + * memory storage alone would keep the object as given. + */ +function jsonRow(row: BulkWriteRow): BulkWriteRow { + return JSON.parse(JSON.stringify(row)) as BulkWriteRow +} + +/** The storage's error object for each rejected row, to its plan's error. */ +const planErrors = new WeakMap() + +/** + * A per-row write error, as RxDB's schema validation reports a document it + * refuses: the row is not written and the response names it. + */ +function writeError( + row: BulkWriteRow, + error: Error, + context: string, +): RxStorageWriteError { + const rejected: RxStorageWriteError = { + status: 422, + isError: true, + context, + documentId: row.document.id, + writeRow: row, + validationErrors: [{ field: `title`, message: error.message }], + schema: schema as never, + } + planErrors.set(rejected, error) + return rejected +} + +/** + * MUT-2's relation for RxDB: a thrown storage error reaches the caller as + * itself, and RxDB reports a write error as an `RxError` whose + * `parameters.writeError` is the storage's error object for that row. + */ +function matchesFailure(reported: unknown, failure: Error): boolean { + if (reported === failure) return true + const rowError = ( + reported as { parameters?: { writeError?: object } } | undefined + )?.parameters?.writeError + return ( + reported instanceof Error && + rowError !== undefined && + planErrors.get(rowError) === failure + ) +} + +type BulkWrite = ( + rows: Array>, + context: string, +) => Promise> + +class RxdbMutationProvider implements MutationProvider { + private readonly plans: Array = [] + private readonly held: Array<() => void> = [] + private readonly log: Array = [] + private foreign = false + /** Each pending write's document as the adapter sent it, before hooks. */ + private readonly sent = new Map() + private storageInstance: RxStorageInstance | undefined + rxCollection: RxCollection | undefined + + /** Memory storage whose `conformance` collection writes go through `write`. */ + readonly storage: ReturnType + + constructor() { + const memory = getRxStorageMemory() + this.storage = { + ...memory, + createStorageInstance: async (params) => { + const instance = await memory.createStorageInstance(params) + if (params.collectionName !== `conformance`) return instance + this.storageInstance = instance as unknown as RxStorageInstance< + Doc, + unknown, + unknown + > + const bulkWrite = instance.bulkWrite.bind( + instance, + ) as unknown as BulkWrite + const write: BulkWrite = (rows, context) => + this.write(rows, context, bulkWrite) + instance.bulkWrite = write as unknown as typeof instance.bulkWrite + return instance + }, + } + } + + private async write( + rows: Array>, + context: string, + bulkWrite: BulkWrite, + ): Promise> { + if (this.foreign) return bulkWrite(rows.map(jsonRow), context) + const response: RxStorageBulkWriteResponse = { error: [] } + for (const row of rows.map(jsonRow)) { + // An insert of a document the storage holds, even deleted, is refused + // as a conflict and has no effect. RxDB's `bulkUpsert` sends one before + // it updates, and answers the conflict itself, so it is not a request. + if (row.previous === undefined && (await this.holds(row.document.id))) { + response.error.push(...(await bulkWrite([row], context)).error) + continue + } + this.log.push(writeRequest(row, this.sent.get(row.document.id))) + this.sent.delete(row.document.id) + const plan = this.plans.shift() ?? acceptAtOnce + if (plan.held) { + await new Promise((resolve) => this.held.push(resolve)) + } + if (plan.outcome === `reject`) { + if (plan.rejectionKind === `thrown`) throw plan.error + response.error.push(writeError(row, plan.error, context)) + continue + } + response.error.push(...(await bulkWrite([row], context)).error) + } + return response + } + + private async holds(id: string): Promise { + const found = await this.storageInstance!.findDocumentsById([id], true) + return found.length > 0 + } + + /** + * RxDB's `preInsert` and `preSave` hooks, which change a document before + * RxDB writes it: the server-side rewrite (`server-rewrites`). Each keeps + * the document as sent for the request, then applies the next plan's + * rewrite. A conflicting insert that RxDB retries as an update runs both + * hooks; the first keeps the document as sent, and rewrites are + * idempotent. + */ + installHooks(rxCollection: RxCollection): void { + const hook = (data: Doc) => { + if (this.foreign || (data as { _deleted?: boolean })._deleted) return + if (!this.sent.has(data.id)) this.sent.set(data.id, { ...data }) + const rewrite = this.plans[0]?.rewrite ?? `none` + const row = rewriteRow(conformanceRow(data), rewrite) + data.title = row.title + if (row.note === undefined) delete data.note + else data.note = row.note + } + rxCollection.preInsert(hook, false) + rxCollection.preSave(hook, false) + } + + plan(plan: WritePlan): void { + this.plans.push(plan) + } + + release(): void { + this.held.shift()?.() + } + + /** Answers every held request, so the database can close. */ + releaseAll(): void { + for (const answer of this.held.splice(0)) answer() + } + + async foreignCommit(change: ProviderChange): Promise { + const rxCollection = this.rxCollection! + this.foreign = true + try { + if (change.type === `upsert`) { + await rxCollection.upsert(document(change.row)) + } else { + await rxCollection.findOne(String(change.id)).remove() + } + } finally { + this.foreign = false + } + } + + /** The provider's starting rows, written before the adapter syncs. */ + async seed(rows: ReadonlyArray): Promise { + this.foreign = true + try { + await this.rxCollection!.bulkInsert(rows.map(document)) + } finally { + this.foreign = false + } + } + + /** The storage's documents, not RxDB's document cache. */ + async rows(): Promise> { + const instance = this.storageInstance! + const { documents } = await instance.query( + prepareQuery(instance.schema, { + selector: { _deleted: false }, + sort: [{ id: `asc` }], + skip: 0, + }), + ) + return documents.map(conformanceRow) + } + + requests(): ReadonlyArray { + return this.log + } +} + +let sequence = 0 + +export function rxdbMutationDriver(): MutationDriver { + return { + adapter: `@tanstack/rxdb-db-collection`, + capabilities: new Set([ + `write-rejection`, + `held-writes`, + `foreign-commits`, + `server-rewrites`, + // `incrementalPatch` keeps the fields an update does not name. + `merged-updates`, + ]), + // A JSON document has no `undefined`: a removed field is absent. + undefinedField: `absent`, + matchesFailure, + rejectionKinds: [`write-error`, `thrown`], + async create(seed): Promise { + const provider = new RxdbMutationProvider() + const db: RxDatabase = await createRxDatabase({ + name: `conformance-mutations-${++sequence}`, + storage: provider.storage, + multiInstance: false, + }) + const { conformance } = await db.addCollections({ + conformance: { schema }, + }) + const rxCollection = conformance as RxCollection + provider.rxCollection = rxCollection + provider.installHooks(rxCollection) + await provider.seed(seed) + const collection = createCollection( + rxdbCollectionOptions({ rxCollection, syncMode: `eager` }), + ) + return { + collection: observedView(collection, { + row: conformanceRow, + key: conformanceKey, + }), + base: () => + [...collection.base.values()].map((doc) => conformanceRow(doc)), + writer: { + insert: (row) => collection.insert(document(row)), + update: (row) => + collection.update(String(row.id), (draft) => { + draft.title = row.title + if (Object.hasOwn(row, `note`)) draft.note = row.note + // Only a held note value is removed. + else if (typeof draft.note === `string`) delete draft.note + }), + delete: (id) => collection.delete(String(id)), + }, + provider, + dispose: async () => { + // Removing the database waits for its writes, so answer any write + // a failed history left held. + provider.releaseAll() + await collection.cleanup() + await db.remove() + }, + } + }, + } +} diff --git a/packages/rxdb-db-collection/tests/mutation-conformance-oracle.property.test.ts b/packages/rxdb-db-collection/tests/mutation-conformance-oracle.property.test.ts new file mode 100644 index 0000000000..2794839626 --- /dev/null +++ b/packages/rxdb-db-collection/tests/mutation-conformance-oracle.property.test.ts @@ -0,0 +1,72 @@ +import { fc } from '@fast-check/vitest' +import { describe, it } from 'vitest' +import { + oraclePropertyOptions, + oracleReplayTest, + oracleRuns, +} from '../../db/tests/oracle-config' +import { declareOracle } from '../../db/tests/oracle-declaration' +import { createOracleReach } from '../../db/tests/oracle-reach' +import { + checkMutationHistory, + mutationHistoryArb, + mutationModel, + mutationProperty, + mutationReachStatesFor, +} from '../../db-adapter-conformance/src' +import { rxdbMutationDriver } from './mutation-conformance-driver' + +/** + * # RxDB on the shared mutation oracle + * + * A receiving driver for `packages/db-adapter-conformance`. The shared + * module states the laws (MUT-1, MUT-2, MUT-8, MUT-10, MUT-13, FID-5, + * HEALTH-5), the model, the history grammar, and the check. This file + * supplies only the RxDB production driver in `mutation-conformance-driver.ts` + * and the campaigns; it states no expected results. + * + * The driver runs a real RxDB database on memory storage, and its provider + * wraps that storage's `bulkWrite`, so it declares every mutation + * capability: the storage can reject or hold a write, and another client can + * write into the database. + */ + +const property = `adapter-conformance.mutations.rxdb` +const driver = rxdbMutationDriver() +const historyArb = mutationHistoryArb(driver) +const runs = 300 +const campaignTimeoutMs = 120_000 + +async function campaign(options: fc.Parameters<[unknown]>) { + const reach = createOracleReach(property, mutationReachStatesFor(driver), { + minimum: 5, + }) + await fc.assert(mutationProperty(driver, reach), options) + return reach +} + +describe(`mutation conformance: RxDB`, () => { + oracleReplayTest(it)( + `conforms on a fixed-seed campaign`, + async () => { + const reach = await campaign({ seed: 2118, numRuns: oracleRuns(runs) }) + reach.assertReached() + }, + campaignTimeoutMs, + ) + + oracleReplayTest(it, property)( + `conforms on a random campaign or a replayed seed`, + async () => { + const reach = await campaign(oraclePropertyOptions(runs, property)) + reach.assertReached() + }, + campaignTimeoutMs, + ) +}) + +declareOracle({ + check: [checkMutationHistory, mutationProperty], + model: [mutationModel.step, mutationModel.visible], + grammar: [historyArb], +}) diff --git a/packages/rxdb-db-collection/tests/rxdb.test.ts b/packages/rxdb-db-collection/tests/rxdb.test.ts index 53314f4542..6e22088c7c 100644 --- a/packages/rxdb-db-collection/tests/rxdb.test.ts +++ b/packages/rxdb-db-collection/tests/rxdb.test.ts @@ -1,5 +1,9 @@ import { describe, expect, it, vi } from 'vitest' -import { createCollection, createTransaction } from '@tanstack/db' +import { + UpdateKeyNotFoundError, + createCollection, + createTransaction, +} from '@tanstack/db' import { addRxPlugin, createRxDatabase, @@ -743,6 +747,31 @@ describe(`RxDB Integration`, () => { }) describe(`error handling`, () => { + it(`rejects an update whose document another client deleted`, async () => { + const { collection, rxCollection, db } = await createTestState([ + { id: `1`, name: `first` }, + ]) + try { + // Another tab deleted the document before the update's handler ran. + const findOne = rxCollection.findOne.bind(rxCollection) + vi.spyOn(rxCollection, `findOne`).mockImplementationOnce((id) => { + const query = findOne(id as string) + return Object.assign(query, { + exec: () => Promise.resolve(null), + }) as unknown as ReturnType + }) + const tx = collection.update(`1`, (draft) => { + draft.name = `second` + }) + // The write is not persisted, so it must not report success. + await expect(tx.when(`settled`)).rejects.toBeInstanceOf( + UpdateKeyNotFoundError, + ) + } finally { + await db.remove() + } + }) + function expectAdditionalPropertyError(error: unknown, id: string) { expect(error).toBeInstanceOf(Error) expect(error).toHaveProperty( diff --git a/packages/trailbase-db-collection/src/trailbase.ts b/packages/trailbase-db-collection/src/trailbase.ts index c4956328fe..01145e1be7 100644 --- a/packages/trailbase-db-collection/src/trailbase.ts +++ b/packages/trailbase-db-collection/src/trailbase.ts @@ -83,6 +83,9 @@ function convertPartial< return Object.fromEntries( Object.keys(input).map((k: string) => { const value = input[k] + // A removed field is an own `undefined`, which JSON would drop; it is + // the column's `NULL`. + if (value === undefined) return [k, null] return [k, c[k]?.(value as any) ?? value] }), ) as OutputType @@ -113,6 +116,11 @@ export interface TrailBaseCollectionConfig< syncMode?: TrailBaseSyncMode parse: Conversions + /** + * Per-field conversions for values sent to TrailBase. An update that + * removes a field sends `null`, which clears the column, without calling + * that field's conversion. + */ serialize: Conversions } @@ -140,7 +148,15 @@ export function trailBaseCollectionOptions< const serialIns = (item: TItem) => convert(config.serialize, item) + // Change events per key, and this client's answered writes per key, + // counted while a write is admitted. Each answered write on a key produces + // one event there, in order, so a write is confirmed by the event its + // answer is numbered for, not by an earlier write's late one (MUT-7). const seenIds = new Store(new Map()) + const answeredIds = new Map() + let eventSequence = 0 + // Writes admitted and not yet settled. + let writes = 0 const internalSyncMode = config.syncMode ?? `eager` let fullSyncCompleted = false @@ -149,8 +165,13 @@ export function trailBaseCollectionOptions< ids: Array, timeout: number = 120 * 1000, ): Promise => { + const tickets = ids.map((id) => { + const ticket = (answeredIds.get(id) ?? 0) + 1 + answeredIds.set(id, ticket) + return [id, ticket] as const + }) const completed = (value: Map) => - ids.every((id) => value.has(id)) + tickets.every(([id, ticket]) => (value.get(id) ?? 0) >= ticket) if (completed(seenIds.state)) { return Promise.resolve() } @@ -171,6 +192,31 @@ export function trailBaseCollectionOptions< }) } + // Writes reach TrailBase in author order: a write starts at once when no + // other is being sent, and otherwise after the earlier one's response + // (MUT-10). It does not wait for the earlier one's change event. A write + // returns the ids it wrote and settles once the change stream carries each + // of them (MUT-7). + let sending = 0 + let turn: Promise = Promise.resolve() + const persist = async (write: () => Promise>) => { + writes++ + try { + const sent = (sending++ === 0 ? write() : turn.then(write)).finally( + () => { + sending-- + }, + ) + turn = sent.catch(() => {}) + await awaitIds(await sent) + } finally { + if (--writes === 0) { + seenIds.setState(() => new Map()) + answeredIds.clear() + } + } + } + let eventReader: ReadableStreamDefaultReader | undefined const cancelEventReader = () => { if (eventReader) { @@ -187,15 +233,10 @@ export function trailBaseCollectionOptions< sync: (params: SyncParams) => { const { begin, write, commit, markReady, fail, collection } = params let cancelled = false - let periodicCleanupTask: ReturnType | undefined const cleanup = () => { cancelled = true settleListener() cancelEventReader() - if (periodicCleanupTask !== undefined) { - clearInterval(periodicCleanupTask) - periodicCleanupTask = undefined - } } // TrailBase cursors are opaque. Cache a cursor by its last fetched key @@ -205,9 +246,8 @@ export function trailBaseCollectionOptions< const cursors = new Map() let listening = false - // Orders this run's change events against its list calls. Only loads - // in flight compare against it, so it is kept only while one runs. - let eventSequence = 0 + // Orders change events against this run's list calls. Only loads in + // flight compare against it, so it is kept only while one runs. const eventSequenceByKey = new Map() let loadsInFlight = 0 // Settles when the listener runs, cannot start (with the failure), or @@ -404,14 +444,15 @@ export function trailBaseCollectionOptions< void commit() if (value) { - ++eventSequence + const sequence = ++eventSequence if (loadsInFlight > 0) - eventSequenceByKey.set(getKey(value), eventSequence) - seenIds.setState((curr: Map) => { - const newIds = new Map(curr) - newIds.set(String(getKey(value)), Date.now()) - return newIds - }) + eventSequenceByKey.set(getKey(value), sequence) + if (writes > 0) { + const id = String(getKey(value)) + seenIds.setState((curr: Map) => + new Map(curr).set(id, (curr.get(id) ?? 0) + 1), + ) + } } } } @@ -431,7 +472,6 @@ export function trailBaseCollectionOptions< } async function start() { - let reader: ReadableStreamDefaultReader | undefined try { const eventStream = await config.recordApi.subscribe(`*`) if (cancelled) { @@ -439,7 +479,7 @@ export function trailBaseCollectionOptions< return } const subscribedReader = eventStream.getReader() - reader = eventReader = subscribedReader + eventReader = subscribedReader // Start listening for subscriptions first. Otherwise, we'd risk a gap // between the initial fetch and starting to listen. @@ -482,37 +522,6 @@ export function trailBaseCollectionOptions< reportStreamEnd(error, true) return } - - // Lastly, start a periodic cleanup task that will be removed when the - // reader closes. - if (cancelled || !reader) return - - periodicCleanupTask = setInterval(() => { - seenIds.setState((curr) => { - const now = Date.now() - let anyExpired = false - - const notExpired = Array.from(curr.entries()).filter(([_, v]) => { - const expired = now - v > 300 * 1000 - anyExpired = anyExpired || expired - return !expired - }) - - if (anyExpired) { - return new Map(notExpired) - } - return curr - }) - }, 120 * 1000) - - const clearCleanupTask = () => { - if (periodicCleanupTask !== undefined) { - clearInterval(periodicCleanupTask) - periodicCleanupTask = undefined - } - } - // listen() reports read errors. Observe this separate promise too. - void reader.closed.then(clearCleanupTask, clearCleanupTask) } void start() @@ -550,59 +559,44 @@ export function trailBaseCollectionOptions< ...collectionConfig, sync, getKey, - onInsert: async (params: InsertMutationFnParams) => { - const ids = await config.recordApi.createBulk( - params.transaction.mutations.map((tx) => { - const { type, modified } = tx - if (type !== `insert`) { - throw new ExpectedInsertTypeError(type) - } - return serialIns(modified) - }), - ) - - // The optimistic mutation overlay is removed on return, so at this point - // we have to ensure that the new record was properly added to the local - // DB by the subscription. - await awaitIds(ids.map((id) => String(id))) - }, - onUpdate: async (params: UpdateMutationFnParams) => { - const ids: Array = await Promise.all( - params.transaction.mutations.map(async (tx) => { - const { type, changes, key } = tx - if (type !== `update`) { - throw new ExpectedUpdateTypeError(type) - } - + // The optimistic mutation overlay is removed when a handler returns, so + // each one waits until the subscription has applied what it wrote. + onInsert: (params: InsertMutationFnParams) => + persist(async () => { + const ids = await config.recordApi.createBulk( + params.transaction.mutations.map((tx) => { + const { type, modified } = tx + if (type !== `insert`) { + throw new ExpectedInsertTypeError(type) + } + return serialIns(modified) + }), + ) + return ids.map((id) => String(id)) + }), + // A transaction's requests go out one at a time in mutation order, so the + // next write starts only after each of them is answered (MUT-8, MUT-10); + // a rejection ends the transaction there. + onUpdate: (params: UpdateMutationFnParams) => + persist(async () => { + const ids: Array = [] + for (const { type, changes, key } of params.transaction.mutations) { + if (type !== `update`) throw new ExpectedUpdateTypeError(type) await config.recordApi.update(key, serialUpd(changes)) - - return String(key) - }), - ) - - // The optimistic mutation overlay is removed on return, so at this point - // we have to ensure that the new record was properly updated in the local - // DB by the subscription. - await awaitIds(ids) - }, - onDelete: async (params: DeleteMutationFnParams) => { - const ids: Array = await Promise.all( - params.transaction.mutations.map(async (tx) => { - const { type, key } = tx - if (type !== `delete`) { - throw new ExpectedDeleteTypeError(type) - } - + ids.push(String(key)) + } + return ids + }), + onDelete: (params: DeleteMutationFnParams) => + persist(async () => { + const ids: Array = [] + for (const { type, key } of params.transaction.mutations) { + if (type !== `delete`) throw new ExpectedDeleteTypeError(type) await config.recordApi.delete(key) - return String(key) - }), - ) - - // The optimistic mutation overlay is removed on return, so at this point - // we have to ensure that the new record was properly updated in the local - // DB by the subscription. - await awaitIds(ids) - }, + ids.push(String(key)) + } + return ids + }), utils: { cancel: cancelEventReader, }, diff --git a/packages/trailbase-db-collection/tests/mutation-conformance-driver.ts b/packages/trailbase-db-collection/tests/mutation-conformance-driver.ts new file mode 100644 index 0000000000..1007c55b3d --- /dev/null +++ b/packages/trailbase-db-collection/tests/mutation-conformance-driver.ts @@ -0,0 +1,259 @@ +/** + * Production driver for the shared mutation oracle + * (`packages/db-adapter-conformance/src/mutation-oracle.ts`). It runs the real + * TrailBase adapter in eager mode against a controlled RecordApi that acts as + * a TrailBase service: it holds the table's rows, answers `list`, `createBulk`, + * `update`, and `delete` by plan, and sends the change stream's events. + * + * The oracle's steps map onto TrailBase like this: + * + * - a record is a SQL row with every column, so an absent `note` is `NULL` + * (`undefinedField: 'null'`); the observed view maps `NULL` back to absent + * (`fromSqlRow`). The driver inserts an absent `note` as `null`, as a + * TrailBase application must (its record type has every column), and + * removes a held `note` by assigning `undefined`, the update API's + * removed field. Removing it by assigning `null` is not generated; + * - a write request is one record of a `createBulk` call, one `update` call, + * which carries only the changed fields and which the service merges into + * its row (`field-updates`), or one `delete` call. Each request is + * recorded as JSON, as the SDK sends it; + * - a plan answers the next request: a rejection rejects the call with the + * plan's error, and a held request waits until `release`. An accepted + * write commits and sends its change event at once, or, with a held echo, + * only at `deliver`, as an event can trail the HTTP response + * (`held-echoes`). The change stream is in commit order, so a held event + * stalls it and later events wait behind it (`log-ordered-echoes`); + * TrailBase confirms each write through its change stream (the law set's + * `confirm-through-sync`); + * - a rewrite (`server-rewrites`) is the service changing the stored row, as + * a column default, trigger, or check does; its event carries the stored + * row; + * - a commit by another client changes the service's rows and sends its + * event at once. + * + * Limits: the record API is a test double that bypasses the SDK's fetch and + * SSE decoding, as `conformance-driver.ts` does; the key is the client's, + * never server-generated. + */ +import { createCollection } from '@tanstack/db' +import { trailBaseCollectionOptions } from '../src/trailbase' +import { + acceptAtOnce, + fromSqlRow, + observedView, + rewriteRow, + toSqlRow, +} from '../../db-adapter-conformance/src' +import { MockRecordApi } from './mock-record-api' +import type { Event } from 'trailbase' +import type { + ConformanceRow, + MutationDriver, + MutationFixture, + MutationProvider, + ProviderChange, + WritePlan, + WriteRequest, +} from '../../db-adapter-conformance/src' + +/** + * A TrailBase record: a SQL row, so it has every column. The application + * writes an absent `note` as `null`, and removes one by assigning + * `undefined`, the update API's removed field (an own `undefined`). + */ +type ConformanceRecord = { + id: number + title: string + note: string | null | undefined +} + +/** A value as the SDK sends it: JSON, so an `undefined` field is absent. */ +const asSent = (value: T): T => JSON.parse(JSON.stringify(value)) as T + +/** A request's record as the request log holds it; `null` is kept. */ +const sentRow = (record: Partial) => + asSent(record) as Partial + +class TrailBaseMutationProvider implements MutationProvider { + readonly api = new MockRecordApi() + /** The service's table: SQL rows. */ + private readonly table = new Map() + private readonly plans: Array = [] + private readonly held: Array<() => void> = [] + /** Events the stream holds at a held echo; `own` marks the holding one. */ + private readonly withheld: Array<{ send: () => void; own: boolean }> = [] + private readonly log: Array = [] + private controller: ReadableStreamDefaultController | undefined + + constructor(seed: ReadonlyArray) { + for (const row of seed) this.table.set(row.id, toSqlRow(row)) + this.api.subscribe.mockImplementation(() => + Promise.resolve( + new ReadableStream({ + start: (controller) => { + this.controller = controller + }, + // Cleanup cancels the stream; later events have no reader. + cancel: () => { + this.controller = undefined + }, + }), + ), + ) + this.api.list.mockImplementation(() => + Promise.resolve({ records: [...this.table.values()].map(asSent) }), + ) + this.api.createBulk.mockImplementation((records) => + Promise.all( + records.map((record) => + this.request({ type: `insert`, id: record.id, row: sentRow(record) }), + ), + ).then(() => records.map((record) => record.id)), + ) + this.api.update.mockImplementation((id, record) => + this.request({ + type: `update`, + id: Number(id), + row: { id: Number(id), ...sentRow(record) }, + }), + ) + this.api.delete.mockImplementation((id) => + this.request({ type: `delete`, id: Number(id) }), + ) + } + + /** One write request: recorded, then answered by the next plan. */ + private request(request: WriteRequest): Promise { + this.log.push(request) + const plan = this.plans.shift() ?? acceptAtOnce + return new Promise((resolve, reject) => { + const answer = () => { + if (plan.outcome === `reject`) { + reject(plan.error) + return + } + this.commit(this.change(request, plan), plan.echoHeld) + resolve() + } + if (plan.held) this.held.push(answer) + else answer() + }) + } + + /** The change an accepted request makes to the service's table. */ + private change(request: WriteRequest, plan: WritePlan): ProviderChange { + if (request.type === `delete`) return { type: `delete`, id: request.id } + // An update merges the fields it sends into the stored row. + const merged = { + ...(request.type === `update` ? this.table.get(request.id) : {}), + ...request.row, + } as Partial + return { + type: `upsert`, + row: rewriteRow(fromSqlRow({ note: null, ...merged }), plan.rewrite), + } + } + + /** Commits a change and sends its event, at once or at `deliver`. */ + private commit(change: ProviderChange, withhold = false): void { + let event: Event + if (change.type === `delete`) { + const record = this.table.get(change.id) + // Deleting a row the table lacks changes nothing and sends no event. + if (!record) return + this.table.delete(change.id) + event = { Delete: asSent(record) } + } else { + const record = toSqlRow(change.row) + const held = this.table.has(change.row.id) + this.table.set(change.row.id, record) + event = held ? { Update: asSent(record) } : { Insert: asSent(record) } + } + const send = () => this.controller?.enqueue(event) + // The change stream is in commit order: an event waits behind a held one. + if (withhold || this.withheld.length > 0) + this.withheld.push({ send, own: withhold }) + else send() + } + + plan(plan: WritePlan): void { + this.plans.push(plan) + } + + release(): void { + this.held.shift()?.() + } + + /** Answers every held request and sends every withheld event. */ + drain(): void { + for (const answer of this.held.splice(0)) answer() + for (const { send } of this.withheld.splice(0)) send() + } + + deliver(): void { + this.withheld.shift()?.send() + while (this.withheld[0] && !this.withheld[0].own) + this.withheld.shift()!.send() + } + + foreignCommit(change: ProviderChange): void { + this.commit(change) + } + + rows(): ReadonlyArray { + return [...this.table.values()].map(fromSqlRow) + } + + requests(): ReadonlyArray { + return this.log + } +} + +export function trailBaseMutationDriver(): MutationDriver { + return { + adapter: `@tanstack/trailbase-db-collection (eager)`, + capabilities: new Set([ + `write-rejection`, + `held-writes`, + `foreign-commits`, + `server-rewrites`, + `field-updates`, + `held-echoes`, + `log-ordered-echoes`, + ]), + undefinedField: `null`, + create(seed): MutationFixture { + const provider = new TrailBaseMutationProvider(seed) + const collection = createCollection( + trailBaseCollectionOptions({ + recordApi: provider.api, + getKey: (row: ConformanceRecord) => row.id, + syncMode: `eager`, + parse: {}, + serialize: {}, + }), + ) + return { + collection: observedView(collection, { + row: fromSqlRow, + key: (key) => key as number, + }), + base: () => [...collection.base.values()].map(fromSqlRow), + writer: { + insert: (row) => collection.insert(toSqlRow(row)), + update: (row) => + collection.update(row.id, (draft) => { + draft.title = row.title + if (Object.hasOwn(row, `note`)) draft.note = row.note + // A `NULL` note is already absent; only a held one is removed. + else if (typeof draft.note === `string`) draft.note = undefined + }), + delete: (id) => collection.delete(id), + }, + provider, + drain: () => provider.drain(), + dispose: () => collection.cleanup(), + } + }, + } +} diff --git a/packages/trailbase-db-collection/tests/mutation-conformance-oracle.property.test.ts b/packages/trailbase-db-collection/tests/mutation-conformance-oracle.property.test.ts new file mode 100644 index 0000000000..42cfa3dd6c --- /dev/null +++ b/packages/trailbase-db-collection/tests/mutation-conformance-oracle.property.test.ts @@ -0,0 +1,75 @@ +import { fc } from '@fast-check/vitest' +import { describe, it } from 'vitest' +import { + oraclePropertyOptions, + oracleReplayTest, + oracleRuns, +} from '../../db/tests/oracle-config' +import { declareOracle } from '../../db/tests/oracle-declaration' +import { createOracleReach } from '../../db/tests/oracle-reach' +import { + checkMutationHistory, + mutationHistoryArb, + mutationModel, + mutationProperty, + mutationReachStatesFor, +} from '../../db-adapter-conformance/src' +import { trailBaseMutationDriver } from './mutation-conformance-driver' + +/** + * # TrailBase on the shared mutation oracle + * + * A receiving driver for `packages/db-adapter-conformance`. The shared + * module states the laws (MUT-1, MUT-2, MUT-7, MUT-8, MUT-10, MUT-13, FID-5, + * HEALTH-5), the model, the history grammar, and the check. This file + * supplies only the TrailBase production driver in + * `mutation-conformance-driver.ts` and the campaigns; it states no expected + * results. + * + * The driver's RecordApi acts as a TrailBase service with SQL rows. It + * declares every mutation capability: the service can reject or hold a + * write, rewrite a stored row, and send a write's change event after it + * answers; another client can write; and updates send only their changed + * fields (the law set's `field-updates`). TrailBase confirms each write + * through its change stream (`confirm-through-sync`, MUT-7). + */ + +const property = `adapter-conformance.mutations.trailbase` +const driver = trailBaseMutationDriver() +const historyArb = mutationHistoryArb(driver) +const runs = 300 +const campaignTimeoutMs = 120_000 + +async function campaign(options: fc.Parameters<[unknown]>) { + const reach = createOracleReach(property, mutationReachStatesFor(driver), { + minimum: 5, + }) + await fc.assert(mutationProperty(driver, reach), options) + return reach +} + +describe(`mutation conformance: TrailBase`, () => { + oracleReplayTest(it)( + `conforms on a fixed-seed campaign`, + async () => { + const reach = await campaign({ seed: 2118, numRuns: oracleRuns(runs) }) + reach.assertReached() + }, + campaignTimeoutMs, + ) + + oracleReplayTest(it, property)( + `conforms on a random campaign or a replayed seed`, + async () => { + const reach = await campaign(oraclePropertyOptions(runs, property)) + reach.assertReached() + }, + campaignTimeoutMs, + ) +}) + +declareOracle({ + check: [checkMutationHistory, mutationProperty], + model: [mutationModel.step, mutationModel.visible], + grammar: [historyArb], +}) diff --git a/packages/trailbase-db-collection/tests/trailbase.test.ts b/packages/trailbase-db-collection/tests/trailbase.test.ts index 4eda28a43e..26283ec3c6 100644 --- a/packages/trailbase-db-collection/tests/trailbase.test.ts +++ b/packages/trailbase-db-collection/tests/trailbase.test.ts @@ -300,15 +300,11 @@ describe(`TrailBase Integration`, () => { const errors: Array = [] const recordUnhandled = (error: unknown) => errors.push(error) const reported = vi.spyOn(console, `error`).mockImplementation(() => {}) - const intervals = vi.spyOn(globalThis, `setInterval`) - const clear = vi.spyOn(globalThis, `clearInterval`) process.on(`unhandledRejection`, recordUnhandled) const collection = createCollection(setUp(recordApi)) try { await collection.preload() - const timer = intervals.mock.results.at(-1)?.value - expect(timer).toBeDefined() const expectedRows = new Map([[1, row]]) if (ending === `buffered-close`) { for (const id of [2, 3]) { @@ -322,7 +318,6 @@ describe(`TrailBase Integration`, () => { await new Promise((resolve) => setTimeout(resolve, 0)) expect(errors).toEqual([]) - expect(clear).toHaveBeenCalledWith(timer) expect(stream.locked).toBe(false) expect(collection.status).toBe(`ready`) expect(stripState(collection.state)).toEqual(expectedRows) @@ -343,8 +338,6 @@ describe(`TrailBase Integration`, () => { await new Promise((resolve) => setTimeout(resolve, 0)) process.off(`unhandledRejection`, recordUnhandled) reported.mockRestore() - intervals.mockRestore() - clear.mockRestore() } }, ) @@ -852,6 +845,55 @@ describe(`TrailBase Integration`, () => { } }) + it(`sends a transaction's updates in order and starts the next write only after they are answered`, async () => { + const recordApi = new MockRecordApi() + const rows = [1, 2, 3].map((id) => ({ id, updated: 0, data: `row-${id}` })) + recordApi.list.mockResolvedValue({ records: rows }) + const stream = new TransformStream() + recordApi.subscribe.mockResolvedValue(stream.readable) + const log: Array = [] + const second = createDeferred() + recordApi.update.mockImplementation((id) => { + log.push(`start ${id}`) + const answer = + id === 1 + ? Promise.reject(new Error(`rejected`)) + : id === 2 + ? second.promise + : new Promise(() => {}) + return answer.finally(() => log.push(`answer ${id}`)) + }) + const collection = createCollection(setUp(recordApi)) + try { + await collection.preload() + // One transaction updates rows 1 and 2; TrailBase rejects row 1. + const batch = collection.update([1, 2], (drafts) => { + for (const draft of drafts) draft.updated = 1 + }) + const batchSettled = batch.when(`settled`).catch(() => {}) + const next = collection.update(3, (draft) => { + draft.updated = 1 + }) + void next.when(`settled`).catch(() => {}) + await batchSettled + for (let turn = 0; turn < 5; turn++) await Promise.resolve() + second.resolve() + await new Promise((resolve) => setTimeout(resolve, 0)) + // Requests go out in mutation order, and the next write's request + // starts only once every request of the earlier transaction is + // answered (MUT-8, MUT-10). + const startsOf = (id: number) => log.indexOf(`start ${id}`) + expect(startsOf(1)).toBe(0) + expect(startsOf(3)).toBeGreaterThan(0) + for (const id of [1, 2]) { + if (startsOf(id) >= 0) + expect(log.indexOf(`answer ${id}`)).toBeLessThan(startsOf(3)) + } + } finally { + await collection.cleanup() + } + }) + it(`local inserts, updates and deletes`, async () => { // Prepare mock API. const recordApi = new MockRecordApi() diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index db794bcdab..2b6880b381 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -318,7 +318,7 @@ importers: version: 5.90.20(react@19.0.0) expo: specifier: ~53.0.26 - version: 53.0.26(@babel/core@7.29.7)(@expo/dom-webview@55.0.3)(@expo/metro-runtime@5.0.5(react-native@0.79.6(@babel/core@7.29.7)(@types/react@19.2.13)(react@19.0.0)))(graphql@16.13.2)(react-native@0.79.6(@babel/core@7.29.7)(@types/react@19.2.13)(react@19.0.0))(react@19.0.0) + version: 53.0.26(@babel/core@7.29.7)(@expo/dom-webview@55.0.3)(@expo/metro-runtime@5.0.5(react-native@0.79.6(@babel/core@7.29.7)(@types/react@19.2.13)(react@19.0.0)))(react-native@0.79.6(@babel/core@7.29.7)(@types/react@19.2.13)(react@19.0.0))(react@19.0.0) expo-constants: specifier: ~17.1.0 version: 17.1.8(expo@53.0.26)(react-native@0.79.6(@babel/core@7.29.7)(@types/react@19.2.13)(react@19.0.0)) @@ -415,7 +415,7 @@ importers: version: 5.90.20(react@19.0.0) expo: specifier: ~53.0.26 - version: 53.0.26(@babel/core@7.29.7)(@expo/dom-webview@55.0.3)(@expo/metro-runtime@5.0.5(react-native@0.79.6(@babel/core@7.29.7)(@types/react@19.2.13)(react@19.0.0)))(graphql@16.13.2)(react-native@0.79.6(@babel/core@7.29.7)(@types/react@19.2.13)(react@19.0.0))(react@19.0.0) + version: 53.0.26(@babel/core@7.29.7)(@expo/dom-webview@55.0.3)(@expo/metro-runtime@5.0.5(react-native@0.79.6(@babel/core@7.29.7)(@types/react@19.2.13)(react@19.0.0)))(react-native@0.79.6(@babel/core@7.29.7)(@types/react@19.2.13)(react@19.0.0))(react@19.0.0) expo-constants: specifier: ~17.1.0 version: 17.1.8(expo@53.0.26)(react-native@0.79.6(@babel/core@7.29.7)(@types/react@19.2.13)(react@19.0.0)) @@ -1332,7 +1332,7 @@ importers: version: 0.4.1(vitest@4.1.12) '@neon/realtime': specifier: 0.1.2 - version: 0.1.2 + version: 0.1.2(ws@8.22.0) '@standard-schema/spec': specifier: ^1.1.0 version: 1.1.0 @@ -1714,8 +1714,8 @@ importers: specifier: ^4.4.3 version: 4.4.3 rxdb: - specifier: 17.1.0 - version: 17.1.0(@angular/core@20.3.34(@angular/compiler@20.3.34)(rxjs@7.8.2)(zone.js@0.15.1))(react@19.2.4)(rxjs@7.8.2)(vue@3.5.28(typescript@5.9.3)) + specifier: 17.6.0 + version: 17.6.0(@angular/core@20.3.34(@angular/compiler@20.3.34)(rxjs@7.8.2)(zone.js@0.15.1))(react@19.2.4)(rxjs@7.8.2)(typescript@5.9.3)(vue@3.5.28(typescript@5.9.3)) rxjs: specifier: '>=7.8.2' version: 7.8.2 @@ -2623,14 +2623,17 @@ packages: peerDependencies: '@babel/core': ^7.0.0-0 - '@babel/runtime@7.28.6': - resolution: {integrity: sha512-05WQkdpL9COIMz4LjTxGpPNCdlpyimKppYNoJ5Di5EUObifl8t4tuLuUBBZEpoLYOmfvIWrsp9fCl0HoPRVTdA==} - engines: {node: '>=6.9.0'} - '@babel/runtime@7.29.2': resolution: {integrity: sha512-JiDShH45zKHWyGe4ZNVRrCjBz8Nh9TMmZG1kh4QTK8hCBTWBi8Da+i7s1fJw7/lYpM4ccepSNfqzZ/QvABBi5g==} engines: {node: '>=6.9.0'} + '@babel/runtime@7.29.7': + resolution: {integrity: sha512-Nq8OhGWiZIZGV6hLHoyAKLLcJihP/xFeBMGJoUrxTX2psI8dCifzLhZISFb+VWS3wFMRDmCGw5R+dOySCqPLhw==} + engines: {node: '>=6.9.0'} + + '@babel/runtime@8.0.5': + resolution: {integrity: sha512-7NK+Lz3spQ52XsUGTxIEVU4jYN2/dIaX8sTxRFAUtYmttYZnVh3aehiihv+/Gb7+duMNHl2OrFcBQRyOHScxpg==} + '@babel/template@7.29.7': resolution: {integrity: sha512-puq+Gf35oI24FeN11LkoUQFqv9uwNeWpxXZi/Ji3rRIoKAzKnxRaZ+Gkj0vKS9ZCiTESfng1N9LyOyXvo+m+Gg==} engines: {node: '>=6.9.0'} @@ -5027,10 +5030,22 @@ packages: cpu: [x64] os: [win32] + '@noble/ciphers@2.1.1': + resolution: {integrity: sha512-bysYuiVfhxNJuldNXlFEitTVdNnYUc+XNJZd7Qm2a5j1vZHgY+fazadNFWFaMK/2vye0JVlxV3gHmC0WDfAOQw==} + engines: {node: '>= 20.19.0'} + '@noble/ciphers@2.4.0': resolution: {integrity: sha512-AnjFn0Jv92laAkvMrghlFZq4qQCIN/4DxFV/eooqtC2YTjB7kBeLMS2T9KJX4Dn+ZVXLOwK0lSgqDtx9gvxtiw==} engines: {node: '>= 20.19.0'} + '@noble/curves@2.0.1': + resolution: {integrity: sha512-vs1Az2OOTBiP4q0pwjW5aF0xp9n4MxVrmkFBxc6EKZc6ddYx5gaZiAsZoq0uRRXWbi3AT/sBqn05eRPtn1JCPw==} + engines: {node: '>= 20.19.0'} + + '@noble/hashes@2.0.1': + resolution: {integrity: sha512-XlOlEbQcE9fmuXxrVTXCTlG2nlRXa9Rj3rr5Ue/+tX+nmkgbX720YHh0VR3hBF9xDvwnb8D2shVGOwNx+ulArw==} + engines: {node: '>= 20.19.0'} + '@noble/hashes@2.4.0': resolution: {integrity: sha512-X5XaVWZIBCT7HHZGm5I7ZQXDwLG+bGXuSrMQAW+7Zvl87h1kmc1ZB1VSRJcpUfoUrGQp4Fkoxm5kZ+Ms+aW+eA==} engines: {node: '>= 20.19.0'} @@ -6055,6 +6070,15 @@ packages: resolution: {integrity: sha512-jU12K5YJUpK7bsPhWUgxoZf0R1fkt6z+AiOu+qHybQgWRmV8VuwZhPsp4N6lwMUkRrNmPOgk2143xNmTXEBjcQ==} engines: {node: ^20.19.0 || ^22.12.0 || >=24.0.0, npm: ^6.11.0 || ^7.5.6 || >=8.0.0, yarn: '>= 1.13.0'} + '@scure/base@2.0.0': + resolution: {integrity: sha512-3E1kpuZginKkek01ovG8krQ0Z44E3DHPjc5S2rjJw9lZn3KSQOs8S7wqikF/AH7iRanHypj85uGyxk0XAyC37w==} + + '@scure/bip32@2.0.1': + resolution: {integrity: sha512-4Md1NI5BzoVP+bhyJaY3K6yMesEFzNS1sE/cP+9nuvE7p/b0kx9XbpDHHFl8dHtufcbdHRUUQdRqLIPHN/s7yA==} + + '@scure/bip39@2.0.1': + resolution: {integrity: sha512-PsxdFj/d2AcJcZDX1FXN3dDgitDDTmwf78rKZq1a6c1P1Nan1X/Sxc7667zU3U+AN60g7SxxP0YCVw2H/hBycg==} + '@shikijs/engine-oniguruma@3.13.0': resolution: {integrity: sha512-O42rBGr4UDSlhT2ZFMxqM7QzIU+IcpoTMzb3W7AlziI1ZF7R8eS2M0yt5Ry35nnnTX/LTLXFPUjRFCIW+Operg==} @@ -6693,9 +6717,6 @@ packages: '@types/chai@5.2.2': resolution: {integrity: sha512-8kB30R7Hwqf40JPiKhVzodJs2Qc1ZJ5zuT3uzw5Hq/dhNCl3G3l83jfpdI1e20BP348+fV7VIL/+FxaXkqBmWg==} - '@types/clone@2.1.4': - resolution: {integrity: sha512-NKRWaEGaVGVLnGLB2GazvDaZnyweW9FJLLFL5LhywGJB3aqGMT9R/EUoJoSRP4nzofYnZysuDmrEJtJdAqUOtQ==} - '@types/common-tags@1.8.1': resolution: {integrity: sha512-20R/mDpKSPWdJs5TOpz3e7zqbeCNuMCPhV7Yndk9KU2Rbij2r5W4RzwDPkzC+2lzUqXYu9rFzTktCBnDjHuNQg==} @@ -7720,8 +7741,8 @@ packages: resolution: {integrity: sha512-yQbXgO/OSZVD2IsiLlro+7Hf6Q18EJrKSEsdoMzKePKXct3gvD8oLcOQdIzGupr5Fj+EDe8gO/lxc1BzfMpxvA==} engines: {node: '>=8'} - broadcast-channel@7.3.0: - resolution: {integrity: sha512-UHPhLBQKfQ8OmMFMpmPfO5dRakyA1vsfiDGWTYNvChYol65tbuhivPEGgZZiuetorvExdvxaWiBy/ym1Ty08yA==} + broadcast-channel@7.4.0: + resolution: {integrity: sha512-Oe0EdXTN+BjH0QatATL1tnK0Qs9aKMSML59nYKOJ8Jk6IMT9Y2O+bJGU6MEA9tD7onvj5HnrwQf8fCzXEBcbDQ==} browser-fs-access@0.35.0: resolution: {integrity: sha512-sLoadumpRfsjprP8XzVjpQc0jK8yqHBx0PtUTGYj2fftT+P/t+uyDAQdMgGAPKD011in/O+YYGh7fIs0oG/viw==} @@ -7942,9 +7963,9 @@ packages: resolution: {integrity: sha512-/rFeCpNJQbhSZjGVwO9RFV3xPqbnERS8MmIQzCtD/zl6gpJuV/bMLuN92oG3F7d8oDEHHRrujSXNUr8fpjntKw==} engines: {node: '>=18'} - commander@14.0.3: - resolution: {integrity: sha512-H+y0Jo/T1RZ9qPP4Eh1pkcQcLRglraJaSLoyOtHxu6AapkjWVCy2Sit1QQ4x3Dng8qDlSsZEet7g5Pq06MvTgw==} - engines: {node: '>=20'} + commander@15.0.0: + resolution: {integrity: sha512-z67u4ZhzCL/Tydu1lJARtEZYWbWaN7oYLHbsuzocr6y4N6WZAagG3RQ4FW61V1/0+jImpj293XfrcYnd1qxtPg==} + engines: {node: '>=22.12.0'} commander@2.20.3: resolution: {integrity: sha512-GpVkmM8vF2vQUkj2LvZmD35JxeJOLCwJ9cUkugyk2nuhbv3+mJvpLYYt+0+USMxE+oj+ey/lJEnhZw75x/OMcQ==} @@ -8249,8 +8270,8 @@ packages: devalue@5.9.4: resolution: {integrity: sha512-sPAT4pztbu6586/hrhOnMKS17IJrvg12mXiSPSS3W5qDeN2RGgvZ0diZCm31dBbnevfVmujNO3IM2wrS4Y2Rhg==} - dexie@4.0.10: - resolution: {integrity: sha512-eM2RzuR3i+M046r2Q0Optl3pS31qTWf8aFuA7H9wnsHTwl8EPvroVLwvQene/6paAs39Tbk6fWZcn2aZaHkc/w==} + dexie@4.4.2: + resolution: {integrity: sha512-zMtV8q79EFE5U8FKZvt0Y/77PCU/Hr/RDxv1EDeo228L+m/HTbeN2AjoQm674rhQCX8n3ljK87lajt7UQuZfvw==} diff@8.0.4: resolution: {integrity: sha512-DPi0FmjiSU5EvQV0++GFDOJ9ASQUVFh5kD+OzOnYdi7n3Wpm9hWWGfB/O2blfHcMVTL5WkQXSnRiK9makhrcnw==} @@ -9327,15 +9348,25 @@ packages: graphemer@1.4.0: resolution: {integrity: sha512-EtKwoO6kxCL9WO5xipiHTZlSzBm7WLT627TqC/uVRd0HKmq8NXyebnNYxDoBi7wt8eTWrUrKXCOVaFq9x1kgag==} - graphql-ws@5.16.2: - resolution: {integrity: sha512-E1uccsZxt/96jH/OwmLPuXMACILs76pKF2i3W861LpKBCYtGIyPQGtWLuBLkND4ox1KHns70e83PS4te50nvPQ==} - engines: {node: '>=10'} + graphql-ws@6.3.0: + resolution: {integrity: sha512-qBAXPtFjWHF3QypDyNcBp8Rycd+rmofxUSJsu3bGI9QMMq7pImKBSLSg9hZ2ojIMbJ54T/LPjKBIeOUoJbNBnQ==} + engines: {node: '>=20'} peerDependencies: - graphql: '>=0.11 <=16' + '@fastify/websocket': ^10 || ^11 + crossws: ~0.3 || ~0.4 + graphql: ^15.10.1 || ^16 || ^17 + ws: ^8 + peerDependenciesMeta: + '@fastify/websocket': + optional: true + crossws: + optional: true + ws: + optional: true - graphql@16.13.2: - resolution: {integrity: sha512-5bJ+nf/UCpAjHM8i06fl7eLyVC9iuNAjm9qzkiu2ZGhM0VscSvS6WDPfAwkdkBuoXGM9FJSbKl6wylMwP9Ktig==} - engines: {node: ^12.22.0 || ^14.16.0 || ^16.0.0 || >=17.0.0} + graphql@17.0.2: + resolution: {integrity: sha512-FRWbddMxfkjiB7z+aQDWIR+E34xo9I8c9mtK2RPv8PmMzKRvrdsreHL/Ui/TmwHJfhHChEtsFPyMHKI+xuarQQ==} + engines: {node: ^22.0.0 || ^24.0.0 || ^25.0.0 || >=26.0.0} h3@2.0.1-rc.20: resolution: {integrity: sha512-28ljodXuUp0fZovdiSRq4G9OgrxCztrJe5VdYzXAB7ueRvI7pIUqLU14Xi3XqdYJ/khXjfpUOOD2EQa6CmBgsg==} @@ -9579,10 +9610,6 @@ packages: resolution: {integrity: sha512-0KI/607xoxSToH7GjN1FfSbLoU0+btTicjsQSWQlh/hZykN8KpmMf7uYwPW3R+akZ6R/w18ZlXSHBYXiYUPO3g==} engines: {node: '>= 0.10'} - is-arguments@1.2.0: - resolution: {integrity: sha512-7bVbi0huj/wrIAOzb8U1aszg9kdi3KN/CyU19CTI7tAoZYEZoL9yCDXpbXN+uPsuWnP02cyug1gleqq+TU+YCA==} - engines: {node: '>= 0.4'} - is-array-buffer@3.0.5: resolution: {integrity: sha512-DDfANUiiG2wC1qawP66qlTugJeL5HyzMpfr8lLK+jMQirGzNod0B12cFB/9q838Ru27sBwfw78/rdoU7RERz6A==} engines: {node: '>= 0.4'} @@ -9905,8 +9932,8 @@ packages: jose@6.2.13: resolution: {integrity: sha512-7TIg10ZE1x82YediN+lcrPNvGMKge9yFHQD+Qho/b8y2adfugU3AsvvHfxV0fzyFllT6l9xCjf4RWY9g/Z9PJQ==} - js-base64@3.7.8: - resolution: {integrity: sha512-hNngCeKxIUQiEUN3GPJOkz4wF/YvdUdbNL9hsBcMQTkKzboD7T/q3OYOuuPZLUE6dBxSGpwhk5mwuDud7JVAow==} + js-base64@3.9.4: + resolution: {integrity: sha512-PtOMXpEGuP0RRiRXsjzHzl44dMHxSu2CPvAhinupR1tBa88me+1DPqsobl7eupn5UukjLkln1ZnAOAOXOrYG0Q==} js-tokens@4.0.0: resolution: {integrity: sha512-RdJUflcE3cUzKiMqQgsCu06FPu9UdIJO0beYbPhHN4k6apgJtifcoCtT9bcxOpYBtpD2kCM6Sbzg4CausW/PKQ==} @@ -10557,8 +10584,8 @@ packages: resolution: {integrity: sha512-I9jwMn07Sy/IwOj3zVkVik2JTvgpaykDZEigL6Rx6N9LbMywwUSMtxET+7lVoDLLd3O3IXwJwvuuns8UB/HeAg==} engines: {node: '>=4'} - mingo@7.2.0: - resolution: {integrity: sha512-UeX942qZpofn5L97h295SkS7j/ADf7Qac8gdRCMBPxi0/1m70aeB2owLFvWbyuMj1dowonlivlVRQVDx+6h+7Q==} + mingo@7.2.4: + resolution: {integrity: sha512-KrxHKzMMguL+G1lBC6YbU5UTXdw2viwVHNTPzNYVfEQfSAoetC3M1a7rlveOksn/96oNy+QwBlIXvx/q6rWyxQ==} miniflare@4.20260317.1: resolution: {integrity: sha512-A3csI1HXEIfqe3oscgpoRMHdYlkReQKPH/g5JE53vFSjoM6YIAOGAzyDNeYffwd9oQkPWDj9xER8+vpxei8klA==} @@ -10844,6 +10871,17 @@ packages: resolution: {integrity: sha512-6eZs5Ls3WtCisHWp9S2GUy8dqkpGi4BVSz3GaqiE6ezub0512ESztXUwUB6C6IKbQkY2Pnb/mD4WYojCRwcwLA==} engines: {node: '>=0.10.0'} + nostr-tools@2.25.2: + resolution: {integrity: sha512-7JAx+brEPmqGz1yzaK9MOxvis/Nl1B38j9hVsBSRDMRmxk5XSc0+5LlPEYbzeiUyk41RURBDcfuMzmaT754HAg==} + peerDependencies: + typescript: '>=5.0.0' + peerDependenciesMeta: + typescript: + optional: true + + nostr-wasm@0.1.0: + resolution: {integrity: sha512-78BTryCLcLYv96ONU8Ws3Q1JzjlAt+43pWQhIl86xZmWeegYCNLPml7yQ+gG3vR6V5h4XGj+TxO+SS5dsThQIA==} + npm-bundled@5.0.0: resolution: {integrity: sha512-JLSpbzh6UUXIEoqPsYBvVNVmyrjVZ1fzEFbqxKkTJQkWBO3xFzFT+KDnSKQWwOQNbuWRwt5LSD6HOTLGIWzfrw==} engines: {node: ^20.17.0 || >=22.9.0} @@ -11564,9 +11602,6 @@ packages: resolution: {integrity: sha512-9u/XQ1pvrQtYyMpZe7DXKv2p5CNvyVwzUB6uhLAnQwHMSgKMBR62lc7AHljaeteeHXn11XTAaLLUVZYVZyuRBQ==} engines: {node: '>= 20.19.0'} - reconnecting-websocket@4.4.0: - resolution: {integrity: sha512-D2E33ceRPga0NvTDhJmphEgJ7FUYF0v4lr1ki0csq06OdlxKfugGzN0dSkxM/NfqCxYELK4KcaTOUOjTV6Dcng==} - redent@3.0.0: resolution: {integrity: sha512-6tDA8g98We0zd0GvVeMT9arEOnTw9qM03L9cJXaCjrip1OO764RDBLBfrB4cwzNGDj5OA5ioymC9GkizgWJDUg==} engines: {node: '>=8'} @@ -11716,9 +11751,9 @@ packages: run-parallel@1.2.0: resolution: {integrity: sha512-5l4VyZR86LZ/lDxZTR6jqL8AFE2S0IFLMP26AbjsLVADxHdhB/c0GUsH+y39UfCi3dzz8OlQuPmnaJOMoDHQBA==} - rxdb@17.1.0: - resolution: {integrity: sha512-XmIGECuc5qzvdkszWkHEE5Ukes1AP9wKUwTxvdfviQ88tSYNiIu9+Zf4Ohhd9oEr/ChUcmfvh2Zs9dwOecbY3A==} - engines: {node: '>=18'} + rxdb@17.6.0: + resolution: {integrity: sha512-VmnCk/IXQ5KoUXw1ZsnsFt/DJI6DCCepE7SkHk44kybyMXczBawPuEAaShLtVQj+xrc9hVEyHOxzgPgIrnjgqQ==} + engines: {node: '>=20'} peerDependencies: '@angular/core': '*' '@preact/signals-core': '*' @@ -12773,9 +12808,6 @@ packages: util-deprecate@1.0.2: resolution: {integrity: sha512-EPD5q1uXyFxJpCrLnCc1nHnq3gOa6DZBocAIiI2TaSCA7VCJ1UJDMagCzIkXNsUYfD1daK//LTEQ8xiIbrHtcw==} - util@0.12.5: - resolution: {integrity: sha512-kZf/K6hEIrWHI6XqOFUiiMa+79wE/D8Q+NCNAWclkyg3b4d2k7s0QGepNjiABc+aR3N1PAyHL7p6UcLY6LmrnA==} - utils-merge@1.0.1: resolution: {integrity: sha512-pMZTvIkT1d+TFGvDOqodOclx0QWkkgi6Tdoa8gC8ffGAAqz9pzPTZWAybbsHHoED/ztMtkv/VoYTYyShUn81hA==} engines: {node: '>= 0.4.0'} @@ -13220,6 +13252,18 @@ packages: utf-8-validate: optional: true + ws@8.22.0: + resolution: {integrity: sha512-Ydggc987+RO0AnWtZ/7Wq9FtNvcrL1b/RO0ud9mWjUPgDrsAAwQSF51sm2hm1XofbU/4jkpGEsLFsZZxU+1DOg==} + engines: {node: '>=10.0.0'} + peerDependencies: + bufferutil: ^4.0.1 + utf-8-validate: '>=5.0.2' + peerDependenciesMeta: + bufferutil: + optional: true + utf-8-validate: + optional: true + xcode@3.0.1: resolution: {integrity: sha512-kCz5k7J7XbJtjABOvkc5lJmkiDh8VhjVCGNiqdKCscmVpdVUpEAyXv1xmCLkQJ5dsHqx3IPO4XW+NTDhU/fatA==} engines: {node: '>=10.0.0'} @@ -13311,8 +13355,8 @@ packages: youch@4.1.0-beta.10: resolution: {integrity: sha512-rLfVLB4FgQneDr0dv1oddCVZmKjcJ6yX6mS4pU82Mq/Dt9a3cLZQ62pDBL4AUO+uVrCvtWz3ZFUL2HFAFJ/BXQ==} - z-schema@12.1.0: - resolution: {integrity: sha512-+wLKQ0M58ATDhGxV1rrFHAvOJ8l1p+3VX43OFPpZ1tpTL67m4auvzHKrRtDpnaZnEiCkhInUoRPxewV6moj0uw==} + z-schema@12.4.6: + resolution: {integrity: sha512-jFh7aWwpGeyBAHFcwVPQ1GtS4d1AwhnTmSBtc/m/WtlA5XqCLlzAXu7rIZhKFXQ/iDfg8vEDQxmfWPh0OEJLAw==} engines: {node: '>=22.0.0'} hasBin: true @@ -13338,9 +13382,7 @@ packages: snapshots: - '@0no-co/graphql.web@1.2.0(graphql@16.13.2)': - optionalDependencies: - graphql: 16.13.2 + '@0no-co/graphql.web@1.2.0': {} '@acemir/cssom@0.9.30': {} @@ -14227,10 +14269,12 @@ snapshots: transitivePeerDependencies: - supports-color - '@babel/runtime@7.28.6': {} - '@babel/runtime@7.29.2': {} + '@babel/runtime@7.29.7': {} + + '@babel/runtime@8.0.5': {} + '@babel/template@7.29.7': dependencies: '@babel/code-frame': 7.29.7 @@ -15144,9 +15188,9 @@ snapshots: '@exodus/bytes@1.7.0': {} - '@expo/cli@0.24.24(graphql@16.13.2)': + '@expo/cli@0.24.24': dependencies: - '@0no-co/graphql.web': 1.2.0(graphql@16.13.2) + '@0no-co/graphql.web': 1.2.0 '@babel/runtime': 7.29.2 '@expo/code-signing-certificates': 0.0.6 '@expo/config': 11.0.13 @@ -15165,8 +15209,8 @@ snapshots: '@expo/ws-tunnel': 1.0.6 '@expo/xcpretty': 4.4.0 '@react-native/dev-middleware': 0.79.6 - '@urql/core': 5.2.0(graphql@16.13.2) - '@urql/exchange-retry': 1.3.2(@urql/core@5.2.0(graphql@16.13.2)) + '@urql/core': 5.2.0 + '@urql/exchange-retry': 1.3.2(@urql/core@5.2.0) accepts: 1.3.8 arg: 5.0.2 better-opn: 3.0.2 @@ -15469,7 +15513,7 @@ snapshots: '@expo/dom-webview@55.0.3(expo@53.0.26)(react-native@0.79.6(@babel/core@7.29.7)(@types/react@19.2.13)(react@19.0.0))(react@19.0.0)': dependencies: - expo: 53.0.26(@babel/core@7.29.7)(@expo/dom-webview@55.0.3)(@expo/metro-runtime@5.0.5(react-native@0.79.6(@babel/core@7.29.7)(@types/react@19.2.13)(react@19.0.0)))(graphql@16.13.2)(react-native@0.79.6(@babel/core@7.29.7)(@types/react@19.2.13)(react@19.0.0))(react@19.0.0) + expo: 53.0.26(@babel/core@7.29.7)(@expo/dom-webview@55.0.3)(@expo/metro-runtime@5.0.5(react-native@0.79.6(@babel/core@7.29.7)(@types/react@19.2.13)(react@19.0.0)))(react-native@0.79.6(@babel/core@7.29.7)(@types/react@19.2.13)(react@19.0.0))(react@19.0.0) react: 19.0.0 react-native: 0.79.6(@babel/core@7.29.7)(@types/react@19.2.13)(react@19.0.0) optional: true @@ -16611,10 +16655,12 @@ snapshots: '@tybys/wasm-util': 0.10.1 optional: true - '@neon/realtime@0.1.2': + '@neon/realtime@0.1.2(ws@8.22.0)': dependencies: base64-js: 1.5.1 postgres-array: 3.0.4 + optionalDependencies: + ws: 8.22.0 '@next/env@16.4.1': {} @@ -16642,8 +16688,16 @@ snapshots: '@next/swc-win32-x64-msvc@16.4.1': optional: true + '@noble/ciphers@2.1.1': {} + '@noble/ciphers@2.4.0': {} + '@noble/curves@2.0.1': + dependencies: + '@noble/hashes': 2.0.1 + + '@noble/hashes@2.0.1': {} + '@noble/hashes@2.4.0': {} '@nodelib/fs.scandir@2.1.5': @@ -17726,6 +17780,19 @@ snapshots: transitivePeerDependencies: - chokidar + '@scure/base@2.0.0': {} + + '@scure/bip32@2.0.1': + dependencies: + '@noble/curves': 2.0.1 + '@noble/hashes': 2.0.1 + '@scure/base': 2.0.0 + + '@scure/bip39@2.0.1': + dependencies: + '@noble/hashes': 2.0.1 + '@scure/base': 2.0.0 + '@shikijs/engine-oniguruma@3.13.0': dependencies: '@shikijs/types': 3.13.0 @@ -18667,8 +18734,6 @@ snapshots: dependencies: '@types/deep-eql': 4.0.2 - '@types/clone@2.1.4': {} - '@types/common-tags@1.8.1': {} '@types/connect@3.4.38': @@ -19049,16 +19114,16 @@ snapshots: '@unrs/resolver-binding-win32-x64-msvc@1.11.1': optional: true - '@urql/core@5.2.0(graphql@16.13.2)': + '@urql/core@5.2.0': dependencies: - '@0no-co/graphql.web': 1.2.0(graphql@16.13.2) + '@0no-co/graphql.web': 1.2.0 wonka: 6.3.5 transitivePeerDependencies: - graphql - '@urql/exchange-retry@1.3.2(@urql/core@5.2.0(graphql@16.13.2))': + '@urql/exchange-retry@1.3.2(@urql/core@5.2.0)': dependencies: - '@urql/core': 5.2.0(graphql@16.13.2) + '@urql/core': 5.2.0 wonka: 6.3.5 '@vitejs/plugin-basic-ssl@2.1.0(vite@7.3.6(@types/node@25.2.2)(jiti@2.7.0)(lightningcss@1.33.0)(sass@1.90.0)(terser@5.51.2)(tsx@4.21.0)(yaml@2.8.1))': @@ -19931,9 +19996,9 @@ snapshots: dependencies: fill-range: 7.1.1 - broadcast-channel@7.3.0: + broadcast-channel@7.4.0: dependencies: - '@babel/runtime': 7.28.6 + '@babel/runtime': 7.29.7 oblivious-set: 2.0.0 p-queue: 6.6.2 unload: 2.4.1 @@ -20187,7 +20252,7 @@ snapshots: commander@13.1.0: {} - commander@14.0.3: + commander@15.0.0: optional: true commander@2.20.3: {} @@ -20453,7 +20518,7 @@ snapshots: devalue@5.9.4: {} - dexie@4.0.10: {} + dexie@4.4.2: {} diff@8.0.4: {} @@ -21155,7 +21220,7 @@ snapshots: expo-asset@11.1.7(expo@53.0.26)(react-native@0.79.6(@babel/core@7.29.7)(@types/react@19.2.13)(react@19.0.0))(react@19.0.0): dependencies: '@expo/image-utils': 0.7.6 - expo: 53.0.26(@babel/core@7.29.7)(@expo/dom-webview@55.0.3)(@expo/metro-runtime@5.0.5(react-native@0.79.6(@babel/core@7.29.7)(@types/react@19.2.13)(react@19.0.0)))(graphql@16.13.2)(react-native@0.79.6(@babel/core@7.29.7)(@types/react@19.2.13)(react@19.0.0))(react@19.0.0) + expo: 53.0.26(@babel/core@7.29.7)(@expo/dom-webview@55.0.3)(@expo/metro-runtime@5.0.5(react-native@0.79.6(@babel/core@7.29.7)(@types/react@19.2.13)(react@19.0.0)))(react-native@0.79.6(@babel/core@7.29.7)(@types/react@19.2.13)(react@19.0.0))(react@19.0.0) expo-constants: 17.1.8(expo@53.0.26)(react-native@0.79.6(@babel/core@7.29.7)(@types/react@19.2.13)(react@19.0.0)) react: 19.0.0 react-native: 0.79.6(@babel/core@7.29.7)(@types/react@19.2.13)(react@19.0.0) @@ -21188,7 +21253,7 @@ snapshots: dependencies: '@expo/config': 11.0.13 '@expo/env': 1.0.7 - expo: 53.0.26(@babel/core@7.29.7)(@expo/dom-webview@55.0.3)(@expo/metro-runtime@5.0.5(react-native@0.79.6(@babel/core@7.29.7)(@types/react@19.2.13)(react@19.0.0)))(graphql@16.13.2)(react-native@0.79.6(@babel/core@7.29.7)(@types/react@19.2.13)(react@19.0.0))(react@19.0.0) + expo: 53.0.26(@babel/core@7.29.7)(@expo/dom-webview@55.0.3)(@expo/metro-runtime@5.0.5(react-native@0.79.6(@babel/core@7.29.7)(@types/react@19.2.13)(react@19.0.0)))(react-native@0.79.6(@babel/core@7.29.7)(@types/react@19.2.13)(react@19.0.0))(react@19.0.0) react-native: 0.79.6(@babel/core@7.29.7)(@types/react@19.2.13)(react@19.0.0) transitivePeerDependencies: - supports-color @@ -21235,7 +21300,7 @@ snapshots: expo-file-system@18.1.11(expo@53.0.26)(react-native@0.79.6(@babel/core@7.29.7)(@types/react@19.2.13)(react@19.0.0)): dependencies: - expo: 53.0.26(@babel/core@7.29.7)(@expo/dom-webview@55.0.3)(@expo/metro-runtime@5.0.5(react-native@0.79.6(@babel/core@7.29.7)(@types/react@19.2.13)(react@19.0.0)))(graphql@16.13.2)(react-native@0.79.6(@babel/core@7.29.7)(@types/react@19.2.13)(react@19.0.0))(react@19.0.0) + expo: 53.0.26(@babel/core@7.29.7)(@expo/dom-webview@55.0.3)(@expo/metro-runtime@5.0.5(react-native@0.79.6(@babel/core@7.29.7)(@types/react@19.2.13)(react@19.0.0)))(react-native@0.79.6(@babel/core@7.29.7)(@types/react@19.2.13)(react@19.0.0))(react@19.0.0) react-native: 0.79.6(@babel/core@7.29.7)(@types/react@19.2.13)(react@19.0.0) expo-file-system@55.0.11(expo@55.0.8)(react-native@0.83.2(@babel/core@7.29.7)(@types/react@19.2.13)(react@19.2.0)): @@ -21250,7 +21315,7 @@ snapshots: expo-font@13.3.2(expo@53.0.26)(react@19.0.0): dependencies: - expo: 53.0.26(@babel/core@7.29.7)(@expo/dom-webview@55.0.3)(@expo/metro-runtime@5.0.5(react-native@0.79.6(@babel/core@7.29.7)(@types/react@19.2.13)(react@19.0.0)))(graphql@16.13.2)(react-native@0.79.6(@babel/core@7.29.7)(@types/react@19.2.13)(react@19.0.0))(react@19.0.0) + expo: 53.0.26(@babel/core@7.29.7)(@expo/dom-webview@55.0.3)(@expo/metro-runtime@5.0.5(react-native@0.79.6(@babel/core@7.29.7)(@types/react@19.2.13)(react@19.0.0)))(react-native@0.79.6(@babel/core@7.29.7)(@types/react@19.2.13)(react@19.0.0))(react@19.0.0) fontfaceobserver: 2.3.0 react: 19.0.0 @@ -21270,7 +21335,7 @@ snapshots: expo-keep-awake@14.1.4(expo@53.0.26)(react@19.0.0): dependencies: - expo: 53.0.26(@babel/core@7.29.7)(@expo/dom-webview@55.0.3)(@expo/metro-runtime@5.0.5(react-native@0.79.6(@babel/core@7.29.7)(@types/react@19.2.13)(react@19.0.0)))(graphql@16.13.2)(react-native@0.79.6(@babel/core@7.29.7)(@types/react@19.2.13)(react@19.0.0))(react@19.0.0) + expo: 53.0.26(@babel/core@7.29.7)(@expo/dom-webview@55.0.3)(@expo/metro-runtime@5.0.5(react-native@0.79.6(@babel/core@7.29.7)(@types/react@19.2.13)(react@19.0.0)))(react-native@0.79.6(@babel/core@7.29.7)(@types/react@19.2.13)(react@19.0.0))(react@19.0.0) react: 19.0.0 expo-keep-awake@55.0.4(expo@55.0.8)(react@19.2.0): @@ -21361,7 +21426,7 @@ snapshots: '@react-navigation/native': 7.1.27(react-native@0.79.6(@babel/core@7.29.7)(@types/react@19.2.13)(react@19.0.0))(react@19.0.0) '@react-navigation/native-stack': 7.9.1(@react-navigation/native@7.1.27(react-native@0.79.6(@babel/core@7.29.7)(@types/react@19.2.13)(react@19.0.0))(react@19.0.0))(react-native-safe-area-context@5.4.0(react-native@0.79.6(@babel/core@7.29.7)(@types/react@19.2.13)(react@19.0.0))(react@19.0.0))(react-native-screens@4.11.1(react-native@0.79.6(@babel/core@7.29.7)(@types/react@19.2.13)(react@19.0.0))(react@19.0.0))(react-native@0.79.6(@babel/core@7.29.7)(@types/react@19.2.13)(react@19.0.0))(react@19.0.0) client-only: 0.0.1 - expo: 53.0.26(@babel/core@7.29.7)(@expo/dom-webview@55.0.3)(@expo/metro-runtime@5.0.5(react-native@0.79.6(@babel/core@7.29.7)(@types/react@19.2.13)(react@19.0.0)))(graphql@16.13.2)(react-native@0.79.6(@babel/core@7.29.7)(@types/react@19.2.13)(react@19.0.0))(react@19.0.0) + expo: 53.0.26(@babel/core@7.29.7)(@expo/dom-webview@55.0.3)(@expo/metro-runtime@5.0.5(react-native@0.79.6(@babel/core@7.29.7)(@types/react@19.2.13)(react@19.0.0)))(react-native@0.79.6(@babel/core@7.29.7)(@types/react@19.2.13)(react@19.0.0))(react@19.0.0) expo-constants: 17.1.8(expo@53.0.26)(react-native@0.79.6(@babel/core@7.29.7)(@types/react@19.2.13)(react@19.0.0)) expo-linking: 7.1.7(expo@53.0.26)(react-native@0.79.6(@babel/core@7.29.7)(@types/react@19.2.13)(react@19.0.0))(react@19.0.0) invariant: 2.2.4 @@ -21467,10 +21532,10 @@ snapshots: react-native-edge-to-edge: 1.6.0(react-native@0.79.6(@babel/core@7.29.7)(@types/react@19.2.13)(react@19.0.0))(react@19.0.0) react-native-is-edge-to-edge: 1.2.1(react-native@0.79.6(@babel/core@7.29.7)(@types/react@19.2.13)(react@19.0.0))(react@19.0.0) - expo@53.0.26(@babel/core@7.29.7)(@expo/dom-webview@55.0.3)(@expo/metro-runtime@5.0.5(react-native@0.79.6(@babel/core@7.29.7)(@types/react@19.2.13)(react@19.0.0)))(graphql@16.13.2)(react-native@0.79.6(@babel/core@7.29.7)(@types/react@19.2.13)(react@19.0.0))(react@19.0.0): + expo@53.0.26(@babel/core@7.29.7)(@expo/dom-webview@55.0.3)(@expo/metro-runtime@5.0.5(react-native@0.79.6(@babel/core@7.29.7)(@types/react@19.2.13)(react@19.0.0)))(react-native@0.79.6(@babel/core@7.29.7)(@types/react@19.2.13)(react@19.0.0))(react@19.0.0): dependencies: '@babel/runtime': 7.29.2 - '@expo/cli': 0.24.24(graphql@16.13.2) + '@expo/cli': 0.24.24 '@expo/config': 11.0.13 '@expo/config-plugins': 10.1.2 '@expo/fingerprint': 0.13.4 @@ -21998,11 +22063,13 @@ snapshots: graphemer@1.4.0: {} - graphql-ws@5.16.2(graphql@16.13.2): + graphql-ws@6.3.0(graphql@17.0.2)(ws@8.22.0): dependencies: - graphql: 16.13.2 + graphql: 17.0.2 + optionalDependencies: + ws: 8.22.0 - graphql@16.13.2: {} + graphql@17.0.2: {} h3@2.0.1-rc.20: dependencies: @@ -22215,11 +22282,6 @@ snapshots: ipaddr.js@1.9.1: {} - is-arguments@1.2.0: - dependencies: - call-bound: 1.0.4 - has-tostringtag: 1.0.2 - is-array-buffer@3.0.5: dependencies: call-bind: 1.0.8 @@ -22575,7 +22637,7 @@ snapshots: jose@6.2.13: {} - js-base64@3.7.8: {} + js-base64@3.9.4: {} js-tokens@4.0.0: {} @@ -23298,7 +23360,7 @@ snapshots: min-indent@1.0.1: {} - mingo@7.2.0: {} + mingo@7.2.4: {} miniflare@4.20260317.1: dependencies: @@ -23575,6 +23637,20 @@ snapshots: normalize-path@3.0.0: {} + nostr-tools@2.25.2(typescript@5.9.3): + dependencies: + '@noble/ciphers': 2.1.1 + '@noble/curves': 2.0.1 + '@noble/hashes': 2.0.1 + '@scure/base': 2.0.0 + '@scure/bip32': 2.0.1 + '@scure/bip39': 2.0.1 + nostr-wasm: 0.1.0 + optionalDependencies: + typescript: 5.9.3 + + nostr-wasm@0.1.0: {} + npm-bundled@5.0.0: dependencies: npm-normalize-package-bin: 5.0.0 @@ -24555,8 +24631,6 @@ snapshots: readdirp@5.0.0: {} - reconnecting-websocket@4.4.0: {} - redent@3.0.0: dependencies: indent-string: 4.0.0 @@ -24795,44 +24869,43 @@ snapshots: dependencies: queue-microtask: 1.2.3 - rxdb@17.1.0(@angular/core@20.3.34(@angular/compiler@20.3.34)(rxjs@7.8.2)(zone.js@0.15.1))(react@19.2.4)(rxjs@7.8.2)(vue@3.5.28(typescript@5.9.3)): + rxdb@17.6.0(@angular/core@20.3.34(@angular/compiler@20.3.34)(rxjs@7.8.2)(zone.js@0.15.1))(react@19.2.4)(rxjs@7.8.2)(typescript@5.9.3)(vue@3.5.28(typescript@5.9.3)): dependencies: - '@babel/runtime': 7.29.2 - '@types/clone': 2.1.4 - '@types/cors': 2.8.19 - '@types/express': 5.0.6 + '@babel/runtime': 8.0.5 '@types/simple-peer': 9.11.9 '@types/ws': 8.18.1 - ajv: 8.18.0 - ajv-formats: 3.0.1(ajv@8.18.0) + ajv: 8.20.0 + ajv-formats: 3.0.1(ajv@8.20.0) array-push-at-sort-position: 5.0.0 as-typed: 1.3.2 - broadcast-channel: 7.3.0 + broadcast-channel: 7.4.0 crypto-js: 4.2.0 custom-idle-queue: 5.0.2 - dexie: 4.0.10 + dexie: 4.4.2 event-reduce-js: 6.0.0 get-graphql-from-jsonschema: 8.1.0 - graphql: 16.13.2 - graphql-ws: 5.16.2(graphql@16.13.2) + graphql: 17.0.2 + graphql-ws: 6.3.0(graphql@17.0.2)(ws@8.22.0) is-my-json-valid: 2.20.6 - js-base64: 3.7.8 + js-base64: 3.9.4 jsonschema-key-compression: 1.7.0 - mingo: 7.2.0 + mingo: 7.2.4 + nostr-tools: 2.25.2(typescript@5.9.3) oblivious-set: 2.0.0 - reconnecting-websocket: 4.4.0 rxjs: 7.8.2 simple-peer: 9.11.1 - util: 0.12.5 - ws: 8.20.0 - z-schema: 12.1.0 + ws: 8.22.0 + z-schema: 12.4.6 optionalDependencies: '@angular/core': 20.3.34(@angular/compiler@20.3.34)(rxjs@7.8.2)(zone.js@0.15.1) react: 19.2.4 vue: 3.5.28(typescript@5.9.3) transitivePeerDependencies: + - '@fastify/websocket' - bufferutil + - crossws - supports-color + - typescript - utf-8-validate rxjs@7.8.2: @@ -26038,14 +26111,6 @@ snapshots: util-deprecate@1.0.2: {} - util@0.12.5: - dependencies: - inherits: 2.0.4 - is-arguments: 1.2.0 - is-generator-function: 1.1.0 - is-typed-array: 1.1.15 - which-typed-array: 1.1.19 - utils-merge@1.0.1: {} uuid@13.0.2: {} @@ -26487,6 +26552,8 @@ snapshots: ws@8.21.0: {} + ws@8.22.0: {} + xcode@3.0.1: dependencies: simple-plist: 1.3.1 @@ -26583,13 +26650,13 @@ snapshots: cookie: 1.1.1 youch-core: 0.3.3 - z-schema@12.1.0: + z-schema@12.4.6: dependencies: punycode: 2.3.1 safe-regex2: 5.1.0 validator: 13.15.35 optionalDependencies: - commander: 14.0.3 + commander: 15.0.0 zimmerframe@1.1.5: {}