diff --git a/docs/contributing/oracle-coverage.md b/docs/contributing/oracle-coverage.md index 6d0cbf1751..c4dde2e427 100644 --- a/docs/contributing/oracle-coverage.md +++ b/docs/contributing/oracle-coverage.md @@ -1858,7 +1858,7 @@ generic visitor behavior outside the named collectors. | Correlated include path identity | `packages/db/tests/query/includes-context-transport-oracle.test.ts` | The fixed path witnesses live in `packages/db/tests/query/includes-context-transport.test.ts`. A flat parent field and nested parent field reach one-level and nested `toArray` results independently; dotted ancestor aliases remain distinct through a grandchild route. Initial results and parent updates have exact public-value checks. A conditional projection gives [`a.b`] and [`a`, `b`] different include results and checks parent-key changes plus later child inserts. Removing the compiler's unique route-key allocator loses a public child result; restoring dotted-string deduplication at either builder site loses a distinct parent value. These fixed witnesses do not establish arbitrary path segments, every recursive source form, or every materialization form. | | Alias scope identity | `packages/db/tests/query/includes-oracle.property.test.ts` (`includes alpha-renaming across lexical scopes`), `packages/db/tests/query/includes-scope-identity-oracle.ts`, `packages/db/tests/query/includes-alias-shadowing-oracle.test.ts`, `packages/db/tests/query/includes-outer-join-route-work-oracle.test.ts`, `packages/db/tests/query/validate-aliases.test.ts`, [issue #1975 review](oracle-reviews/issue-1975-scope-identity.md) | Four topologies place a source beside a same-named source: a `from()` subquery with an include, a joined `unionAll()` branch with an include, a collapsed top-level pure wrapper with a join subquery, and an include directly on a joined `unionAll()`, which may reuse a branch alias. The grammar crosses zero to two joins, LEFT or INNER, zero to two predicates, plain, ordered-and-limited, or DISTINCT bodies, include forms, and eager or on-demand finite providers, after preload and after source writes. Includes read their source directly, through a join, through a `unionAll()` with an anchor join, or through a `from()` subquery that reads the parent row. One in four scenarios draws any naming; same-scope and union-branch repeats are rejected while ancestor shadowing is admitted. Each checkpoint compares the canonical naming with plain recomputation, the generated naming with the canonical naming, and per-Collection `loadSubset` WHERE clauses. The exact-output companion independently recomputes parent-child rows for direct, QueryRef, union, and implicit-join forms across two equality operand orders, shadowed and renamed aliases, eager and on-demand children, initial publication and parent/child writes. The recursive forms keep a required outer lockId correlation and an independent captured lockName predicate; the union has a nonempty right branch and a later two-branch overlap. It checks enumerable user and virtual key names, user values, multiplicity, sibling helper reuse, a fresh parent/inner QueryRef declaration across a source update, direct, child union-branch, parent union-branch, parent QueryRef, and nested-include rejection of ancestor/descendant builder reuse, one nested grandparent capture, functional callback child input, grouping, captured whole-row and nested-path spreads across shadowed and renamed aliases, eager and on-demand children, and parent/child writes, and query identity. Focused follow-up cells check same-path group keys under shadowing and renaming through parent and child writes, alpha-renamed pure wrappers at public-row and optimized-identity cuts, and indexed lazy demand for direct or wrapped nested includes through anchor and user writes. Wrong-query controls that drop either inner predicate or substitute a parent-shaped callback value fail at the initial public-row comparison. A finite follow-up crosses direct includes and joined QueryRefs with shadowed or renamed aliases, eager or on-demand sources, and child IDs equal to or distinct from an outer filter literal; it checks public rows and on-demand child admission after preload. Another follow-up projects a captured parent rank through a child QueryRef and checks public rows after preload, a child-rank change, and a parent-rank change. The original compiler misrouted the outer filter in four cases, and the original optimizer lost the captured binding in two cases at their first public-row comparison. Every pinned witness and both campaigns fail on `18abceee4`; each of the four optimizer re-mint sites fails a witness when reverted. Arbitrary nested includes, include-inside-subquery forms beyond the checked LEFT joined QueryRef, other join-subquery topologies, `having`, outer joins beyond the bounded RIGHT/FULL matrix, publication events, unload, ordered windows, other lazy join targets, and temporal demand remain outside these owners; they need witnesses before an alias-scope closure claim covers them. Ancestor/descendant builder reuse is rejected before correlation extraction; one helper builder remains legal across a parent QueryRef, union branch, or joined QueryRef and a sibling include child. Six placement/source-mode cells check construction, exact rows after preload, and a source update; the joined placement includes an unmatched manager that would expose a missing helper join. The original source-tree guard rejects all six. A discarded enumeration of a captured proxy cannot change the selected query identity or rows before or after a child update; the retained-marker implementation fails that identity check. The [PR #2079 follow-up](oracle-reviews/issue-2072-alias-shadowing-followup.md) records its RED trace, the repaired spread and nested-QueryRef cells, and the remaining compiler and optimizer evidence gaps. The [ad5e0108 review](oracle-reviews/issue-2072-alias-shadowing-ad5e0108.md) adds a self-join with an inner alias matching the outer joined source, a joined child whose local source competes with inherited parent context in matched and unmatched LEFT joins, captured/local spreads, grandparent whole-row capture, and direct captured-alias selection. These finite cells compare public rows after preload and selected writes; an alias-only lazy-target mutant fails the eager self-join comparison. The reported on-demand failure did not reproduce with the controlled source. A further finite matrix crosses nested FROM or JOIN placement, field or whole-row capture of a shadowed outer joined alias, shadowed or renamed spelling, and eager or on-demand acquisition. It compares source-role rows after preload, child deletion and movement, joined-row update and movement, and parent update; on-demand cases compare nonempty request sets. Disabling projected binding lookup fails all eight cells at the initial public-row assertion; an alias-only lazy-target mutant survives this matrix, so it adds no broader lazy-target claim. A bounded outer-join extension crosses RIGHT and FULL joins, main- or joined-side correlation, direct and QueryRef joined sources, shadowed and renamed aliases, eager and on-demand sources, two parent routes, and seven initial or post-write cuts. Its independent source-role model checks exact public rows, including both absent sides; on-demand cells also require source acquisition. A join-operand extension crosses an ancestor term combined with a local main or joined term, or serving as the whole main key, with equality reversal, alias renaming, source mode, and five public-row cuts. The [outer-join follow-up](oracle-reviews/issue-2072-alias-shadowing-outer-joins.md) records RED and GREEN evidence. A foreign captured join operand must reject: a finite provenance matrix checks root and include placements, matching or different alias spelling, and eager or on-demand sources at construction or preload settlement; valid local and captured-ancestor operands remain controls. A two-LEFT-join history now checks that a later routed QueryRef or captured operand cannot restore an absent shadowed local source, across alias renaming, eager/on-demand mode, two parents, and local and second-joined arrival/removal. It also selects and updates a parent label while joins are matched, checking that the main route retains that ancestor value. A nested joined QueryRef control projects its join key from an inner tag source while the post alias shadows or renames an outer user alias; an independent plain-row model and installed on-demand post/tag rows are compared at preload, a user-key move, and a tag update. A wrong Collection/SourceId pair loads an irrelevant post at preload and fails the installed-row check. Other nested source shapes and real-provider request schedules remain outside this control. The route-work owner compares public rows and assembled parent-route work as irrelevant joined rows grow for direct main-correlated RIGHT/FULL joins in both source modes; an unconditional-route mutant grows from two to six pairs at preload. RIGHT joins correlated on main now join the outer-join row matrix, beside joined-side correlation and FULL controls. An include child rejects a bound join operand declared inside its parent QueryRef or union branch, across matching/renamed aliases and eager/on-demand sources; the original source-tree collector resolves preload instead of rejecting. A joined QueryRef rejects a captured source declared beside it in the enclosing query: the matrix crosses root/include placement, matching/renamed inner alias, and eager/on-demand sources; four true-ancestor controls assert rows, and the permissive sibling-scope mutant fails all eight invalid cells while those controls pass. The primary oracle now extends the hidden-source rejection through two QueryRef or union levels, with matching/renamed aliases and both source modes; a source-tree-binding mutant fails all eight added cells at the rejection cut. An unrelated bound reference is rejected in WHERE, GROUP BY, HAVING, ORDER BY, direct SELECT, and conditional SELECT across matching/renamed aliases; legal local controls and a grouped include with captured ancestor predicates, HAVING, ordering, shadowing, and parent/child writes prevent a blanket rejection. It also rejects foreign references moved into extracted include parent filters or into the removed correlation equality, each across alias spelling and source mode with valid parent controls. Disabling expression validation fails twelve non-join cells; omitting parent-filter or extracted-correlation validation fails four cells each. A separate plain relational model checks RIGHT-to-LEFT, RIGHT-to-RIGHT, FULL-to-FULL, and FULL-to-LEFT two-join chains under shadowing/renaming, both source modes, main/joined/tag correlation, and six source or ancestor writes; a later-FULL-as-LEFT mutant fails two initial public-row comparisons. The route-work owner now compares public rows and parent-route assembly counts at preload and four main-source writes while irrelevant joined rows grow. Other expression forms, arbitrary join-subquery and lazy-target shapes, deeper captures, other chained outer joins, real-provider schedules, and broader source-work bounds remain open under this owner. The alias-scope owner also checks outer grouping through a nested QueryRef and joined source with shadowed or renamed inner alias, eager or on-demand acquisition, and four source/tag writes at five public-row cuts. The tag insertion duplicates one joined source and the plain model counts both copies. A controlled rewrite probe reached an unbound pushed WHERE inside the QueryRef while the bound GROUP BY remained outside; an inner GROUP BY blocked that pushdown. A matcher mutant that ignored binding IDs failed two same-path group cells at the first public-row cut; a dropped-pushed-predicate mutant failed all four new cells by publishing an extra group. This refutes the reported WHERE-rewrite-to-GROUP-BY-matcher route in those shapes; another lost-provenance route needs a concrete fixture. Base/head work probes confirmed an extra pure-wrapper stage after #2079, but the same full-source outer-WHERE request and keyed lazy-join request on both revisions. No wrapper-stage work budget or safe binding remap is established. Bound root refs caused two route-metadata symbol checks per evaluation on the current head versus zero on base, while captured-ancestor rows remain protected by the primary oracle; a compile-time fast path needs a justified work budget and a preservation witness. The same function-form subquery placed twice compiled two results and invoked its callback four times for two source rows on both revisions. Cache comparison read more query properties on the current head, and a stateful callback produced placement-specific values. A callback identity or purity rule and work budget are needed before cache-equivalence changes. | | Join equality and cold acquisition | `packages/db/tests/query/cold-join-reconciliation-oracle.test.ts` | Independent recomputation for cold acquisition plus direct join/predicate equivalence across established equality domains. Binary/string and nullish classes, replacement histories, raw on-demand values, and both scan/auto-index paths are explicit; the compound lane also covers tuple equality, term order, all four join types, controlled cold sources, and keyed histories. Positive acquisition witnesses require a satisfiable tuple. | -| Aggregate selects | [aggregate select oracle](https://github.com/TanStack/db/blob/main/packages/db/tests/query/aggregate-select-oracle.property.test.ts) | Generated select shapes of aggregate queries with and without `groupBy` by `c.k`, the pair `c.k`, `gt(c.v, 1)`, the triple that adds `gt(c.v, 2)`, `upper(c.k)`, `inArray(c.k, ['a', 'b'])`, a Date-valued `coalesce` or a `NaN`-valued `add`, rebuilt per use (literals, parent fields, the group key, the raw field `c.k`, source fields, `count`/`sum`/`max`, `add`, `caseWhen` with expression or object branches, nested objects, spreads, a nested include) and a HAVING condition that may compare an aggregate, selected as a field or not, at the top level, directly or through a `unionAll`, and in an include whose source may shadow the parent alias, through parent updates and source inserts, updates (including key changes that move a row to another group and route) and deletes. A shape throws `NonAggregateExpressionNotInGroupByError` exactly when a non-key source field appears outside an aggregate in its select or HAVING, or the shape spreads a row or holds a nested include; otherwise every published group equals the model and keeps the select's field order. The same grammar checks the opt-in operator laws: a grouped query whose select and HAVING hold no aggregate throws `GroupingOperatorRequiredError` when its keys are passed bare, and works through `groupKeys()`; a bare query builder throws `IncludeOperatorRequiredError` as a field, inside a nested object, or in a conditional branch; a wrapped include inside a nested object throws (a conditional branch may be an include; the includes-collection oracle publishes one, and a compiler that rejects one fails both that oracle and this one). When several errors apply, any one of them passes. Includes are wrapped in `toArray()`, `include()`, or `materialize()`. In three generated histories of six, the driver strips one kind of operator code, the grouping compiler, the include runtime, or the grouping compiler of the `orderBy` aggregate only, from the IR it built. One history in three is a forced history that strips the grouping compiler everywhere or from `orderBy` only, with no, one, or two group keys in turn, so every campaign reaches each path and compiles it through `new BaseQueryBuilder(ir)`; IR without its operator code must throw the coded error for the missing operator. A runtime-stripped history gives its include a child query that is valid on its own, so other errors cannot hide the missing runtime. Half the grouping-stripped histories give the top-level query a select of `count()` alone, with an aggregate HAVING when the generated shape has one, so a model error set of exactly `groupingRequired` witnesses the grouped, one-group, and HAVING paths. The top-level and include queries may order their groups with one or two `orderBy` calls, each on an aggregate or on a number or string select field through `$selected`, strings with the documented `en-US` collation: published rows and each parent's `kids` fall into blocks of equal values in that order. An `orderBy` aggregate that no top-level field selects throws `AggregateFunctionNotInSelectError` (now stated in the guide's "Ordering by SELECT Fields"), and a stripped `orderBy` aggregate compiles with the compiler that `groupKeys()` or a select or HAVING aggregate brings. One history in four takes ordered shapes from a grammar whose shapes compile, because generated ordered shapes compile in about one case in forty; the source starts with a key `A`, where locale and code-unit order differ. The property declares twenty-one states with `createOracleReach`, and each campaign of 160 histories reaches each of them; without the forced select, the one-group and HAVING states were never reached, and a mutant that throws the wrong error class only on the implicit path passed three campaigns. Pinned witnesses in `hand-built-ir-operators.test.ts` cover the old `IncludesSubquery` argument order, an aggregate only in `orderBy` without a select aggregate, and the mixed `orderBy` replay. The compiler before the order step took the query's grouping compiler fails the generated `orderBy` dimension (`top compiles`); an order step that ignores direction, keeps only the first `orderBy` call, or sorts strings by code unit, an include query that ignores direction, a group step that keeps only the first one or two keys, and a check that throws the wrong error class only beside HAVING each fail it (the last passed two of three campaigns before the alone states). A mutant without the include-runtime check, which publishes `kids: null`, and one without the grouping guard, which throws a `TypeError`, both fail it. A mutant that skips the grouping-operator check fails it with a `TypeError`, and one that treats a bare builder as an include fails it with the wrong error class. Open: an `orderBy` term on a raw expression other than an aggregate, and on boolean, Date, `NaN`, or object fields, whose order the docs do not state. The operator laws are generated only for aggregate and grouped selects; for plain selects, `includes.test.ts` pins a bare builder as a field, nested, and in a conditional, because this model publishes one row per group and needs a separate row-per-source model for them. Open: the random campaign's declared reach of "a string field orders A against a lowercase key" can miss on an unlucky seed. One full run on 2026-10-28 missed it (0 histories at 160 runs); 15 reruns reached it. The reach check runs after the campaign, so the miss records no replayable seed. Needed: a grammar weight that makes the case common, or a reported seed for a missed reach. | +| Aggregate selects | [aggregate select oracle](https://github.com/TanStack/db/blob/main/packages/db/tests/query/aggregate-select-oracle.property.test.ts) | Generated select shapes of aggregate queries with and without `groupBy` by `c.k`, the pair `c.k`, `gt(c.v, 1)`, the triple that adds `gt(c.v, 2)`, `upper(c.k)`, `inArray(c.k, ['a', 'b'])`, a Date-valued `coalesce` or a `NaN`-valued `add`, rebuilt per use (literals, parent fields, the group key, the raw field `c.k`, source fields, `count`/`sum`/`max`, `add`, `caseWhen` with expression or object branches, nested objects, spreads, a nested include) and a HAVING condition that may compare an aggregate, selected as a field or not, at the top level, directly or through a `unionAll`, and in an include whose source may shadow the parent alias, through parent updates and source inserts, updates (including key changes that move a row to another group and route) and deletes. A shape throws `NonAggregateExpressionNotInGroupByError` exactly when a non-key source field appears outside an aggregate in its select or HAVING, or the shape spreads a row or holds a nested include; otherwise every published group equals the model and keeps the select's field order. The same grammar checks the opt-in operator laws: a grouped query whose select and HAVING hold no aggregate throws `GroupingOperatorRequiredError` when its keys are passed bare, and works through `groupKeys()`; a bare query builder throws `IncludeOperatorRequiredError` as a field, inside a nested object, or in a conditional branch; a wrapped include inside a nested object throws (a conditional branch may be an include; the includes-collection oracle publishes one, and a compiler that rejects one fails both that oracle and this one). When several errors apply, any one of them passes. Includes are wrapped in `toArray()`, `include()`, or `materialize()`. In three generated histories of six, the driver strips one kind of operator code, the grouping compiler, the include runtime, or the grouping compiler of the `orderBy` aggregate only, from the IR it built. One history in four is a forced history that strips the grouping compiler everywhere or from `orderBy` only, with no, one, or two group keys in turn, to sample each path through `new BaseQueryBuilder(ir)`; IR without its operator code must throw the coded error for the missing operator. A runtime-stripped history gives its include a child query that is valid on its own, so other errors cannot hide the missing runtime. Half the grouping-stripped histories give the top-level query a select of `count()` alone, with an aggregate HAVING when the generated shape has one, so a model error set of exactly `groupingRequired` witnesses the grouped, one-group, and HAVING paths. The top-level and include queries may order their groups with one or two `orderBy` calls, each on an aggregate or on a number or string select field through `$selected`, strings with the documented `en-US` collation: published rows and each parent's `kids` fall into blocks of equal values in that order. An `orderBy` aggregate that no top-level field selects throws `AggregateFunctionNotInSelectError` (now stated in the guide's "Ordering by SELECT Fields"), and a stripped `orderBy` aggregate compiles with the compiler that `groupKeys()` or a select or HAVING aggregate brings. One history in four takes ordered shapes from a grammar whose shapes compile, because generated ordered shapes compile in about one case in forty; the source starts with a key `A`, where locale and code-unit order differ. The property declares twenty-one operator states with `createOracleReach`. Each fixed and random campaign first executes twelve complete structural histories through the shared model, driver and comparison, then 160 generated histories. One reach requires all twenty-one states in the structural set. This guarantee concerns initial publication or compilation only: these inputs have no mutation steps, shadowed alias, or union. A separate generated reach requires the 160 generated histories themselves to enter all twenty-one states and seven additional checkpoints after parent updates, source inserts, updates, deletes, source route moves, a shadowed child alias comparison, and a union-source comparison. Those checkpoints count only after the public model comparison succeeds. Generated coverage remains sampled and can miss a state for a seed; a miss fails and retains the chosen seed and counts. The seven checkpoints do not require every operator state after every mutation or on every route; without the forced select, the one-group and HAVING states were never reached, and a mutant that throws the wrong error class only on the implicit path passed three campaigns. Pinned witnesses in `hand-built-ir-operators.test.ts` cover the old `IncludesSubquery` argument order, an aggregate only in `orderBy` without a select aggregate, and the mixed `orderBy` replay. The compiler before the order step took the query's grouping compiler fails the generated `orderBy` dimension (`top compiles`); an order step that ignores direction, keeps only the first `orderBy` call, or sorts strings by code unit, an include query that ignores direction, a group step that keeps only the first one or two keys, and a check that throws the wrong error class only beside HAVING each fail it (the last passed two of three campaigns before the alone states). A mutant without the include-runtime check, which publishes `kids: null`, and one without the grouping guard, which throws a `TypeError`, both fail it. A mutant that skips the grouping-operator check fails it with a `TypeError`, and one that treats a bare builder as an include fails it with the wrong error class. Open: an `orderBy` term on a raw expression other than an aggregate, and on boolean, Date, `NaN`, or object fields, whose order the docs do not state. The operator laws are generated only for aggregate and grouped selects; for plain selects, `includes.test.ts` pins a bare builder as a field, nested, and in a conditional, because this model publishes one row per group and needs a separate row-per-source model for them. The former seed-dependent string-order reach gap recurred in the main release run on 2026-11-02. The structural histories guarantee the initial premises. They cannot satisfy the separate generated reach. The model, comparisons, reach minimum, fixed seed 2094, and 160 generated runs per campaign remain unchanged. Generated history choices now use named fields rather than a positional tuple. Shared parameter helpers report each selected seed before execution, so a final reach failure after fast-check success has a seed-only campaign replay. Statistics use the exact value `TANSTACK_DB_ORACLE_STATISTICS=1` and survive comparison or reach failures. Removing the structural string history fails the fixed-reach check even when generated histories reach strings. Selecting code-unit order instead of the documented locale order fails the ordinary published-row comparison; seed 2094/path 11 replays that failure directly. The [original repair record](oracle-reviews/2026-11-02-aggregate-select-reach.md) preserves historical evidence. The [nine-item review record](oracle-reviews/2026-11-02-aggregate-select-review.md) records separate counts, generator ablations, replay diagnostics, reproducible commands, and remaining limits. | | Query optimizer semantics | `packages/db/tests/query/optimizer-semantics-oracle.test.ts`, [aggregate review](oracle-reviews/2026-09-28-optimizer-aggregate-pushdown.md), [QueryRef repair](oracle-reviews/2026-09-30-queryref-publication-repair.md), [residual review](oracle-reviews/2026-10-01-pr-1977-residual-convergence.md) | Independent sum recomputation and a materialized-Collection formulation check the first public snapshot of a nested aggregate under a left join. Direct, arithmetic-wrapped, and conditional aggregates distinguish safe from unsafe pushdown; a grouped-key control covers the grouped boundary. Matching and nonmatching global aggregates also cross a nested QueryRef inner join with absent, accepting, and rejecting outer predicates. The matching initial cases fail on the pre-repair revision and pass after the computed-projection lookup repair. A separate fixed residual-convergence grammar checks exact predicate and outer WHERE clause counts at `optimizeQuery` return and after re-optimization: LEFT, RIGHT, INNER, and FULL joins; equality, greater-than, nullable-field `isUndefined`, same-source OR, cross-source equality, clause order, separate/AND form, two active predicates, two residual sources, and an existing residual marker. Plain-array LEFT/RIGHT and three-source join models check first public rows, including unmatched rows. Arbitrary expressions, more complex join chains, and incremental optimizer/publication histories remain outside this owner. | | QueryRef operators and user-value boundaries | `packages/db/tests/query/subquery-user-value-oracle.test.ts`, shared `subquery-literal-value-oracle.ts`, supporting `subquery-literal-value-driver.ts`, correlated `subquery-literal-correlated-oracle.test.ts` / `subquery-literal-correlated-oracle.ts` with concrete factory/driver support, supplemental `subquery-user-value.test.ts` and `transactions.test.ts`, [repair record](oracle-reviews/2026-09-30-queryref-publication-repair.md), [chunk 19](oracle-reviews/2026-10-28-pr-2171-oracle-chunks.md) | The independent relational models own joined DISTINCT with outer WHERE, aggregate-source updates and joined findOne histories. Other fixed container/no-select/functional/predicate-literal examples are supplemental ordinary tests, not independent oracle scope. Chunk 19 adds one ordinary-JavaScript literal model with 17 forms × two positions × root/QueryRef receivers: 68 histories and three cuts each. It checks complete ordinary rows, event bags/current/previous values, event-time reads and immutable QueryRef callback inputs, with all 204 receiving and 102 callback states reached once. Strict exact-driver admission passes; unchanged production has eight undefined-to-null REDs (P11), ten empty-container-loss REDs (P12), 50 passing new histories and six passing relational neighbors. Twenty-four hostile and four lawful controls pass; partial selections still fail mandatory reach. Chunk 20 receives the same literal law through toArray, array/singleton materialize and Collection facades: 272 exact histories, all 3,264 path/cut states once, 200 main passes and 72 intended P11/P12 REDs; 33 checker controls and six independent facet controls pass. Legal QueryRef children have their own correlation WHERE and preserve the authored routeGroup input until actual projection removes it. Facade layouts are ordered, startup callbacks permit canonical empty values but owe complete ready inputs, and all 272 titles remain distinct for precise selection. Chunk 21 adds a separate source-public type judgment (`subquery-literal-types-oracle.ts` / `.test.ts`, supporting `subquery-literal-types-support.ts`) for those exact 340 authored receivers: 1,870 actual QueryResult, Collection-row, contextual-input, inline and facade/API markers. All consumers compile without diagnostics under TypeScript 5.9.3, and all 2,550 syntax/admission/marker states execute once. Unchanged production has 280 intended RED histories: 40 raw-nullish precision failures and 272 contextual-input or propagated-inline metadata failures, with 32 overlapping; 60 native histories and 55 checker/source controls pass. Independent review exposed erased call/construct signatures on records; the recorder now retains those declared capabilities and eight existing empty/nonempty × root/nested record controls reject invented signatures, after all eight failed under the original checker. Only the separate four virtual fields at named query-output or published/source Collection row roots are excluded from this ordinary-value judgment; callbacks and inline children remain strict. The full static callback metadata grammar remains open under existing `virtual-row-fields-types-oracle.test.ts`, with the literal sources as receiving witnesses of its accepted P10 law. Source-public inference does not establish declaration portability; an exported inferred factory currently gives TS4058 and is preserved for O22, including exported Collection and built/installed controls. Broader mixed-expression admission, sparse authored arrays, symbols, ordinary-record readonly modifiers, descriptor flags, arbitrary recursion and provider/framework schedules remain open with their sequential receiving owners. Metadata attribution stays with its separate owner; the four reserved fields are excluded only at each observed root row. No production repair or closure is claimed. | | Opaque backend pagination | [window oracle](https://github.com/TanStack/db/blob/main/packages/query-db-collection/tests/cursor-pagination.oracle.test.ts), [cache histories](https://github.com/TanStack/db/blob/main/packages/query-db-collection/tests/cursor-pagination.cache-oracle.test.ts), [cache publication](https://github.com/TanStack/db/blob/main/packages/query-db-collection/tests/cursor-pagination.publication-oracle.test.ts), [browser acquisition boundaries](https://github.com/TanStack/db/blob/main/packages/query-db-collection/tests/cursor-pagination.boundary-oracle.test.ts), [QueryCollection integration](https://github.com/TanStack/db/blob/main/packages/query-db-collection/tests/cursor-pagination-oracle.integration.test.ts) | Full filter/sort/slice reference, opaque token transport, actual Query cache expiry/invalidation/GC, forced refresh during growth, protocol failure publication/recovery, bounded slice work, nested cancellation/replacement, reader abort, browser retry defaults, manual-write cache isolation, and production window publications. Stable backend sequences; not snapshot guarantees for changing endpoints. Peek-ahead remains enabled. | diff --git a/docs/contributing/oracle-reviews/2026-11-02-aggregate-select-reach.md b/docs/contributing/oracle-reviews/2026-11-02-aggregate-select-reach.md new file mode 100644 index 0000000000..2f38239ad7 --- /dev/null +++ b/docs/contributing/oracle-reviews/2026-11-02-aggregate-select-reach.md @@ -0,0 +1,154 @@ +# Aggregate select structural reach, 2026-11-02 + +This is the historical record for source `bf90c7266c23765220384d5a027de1810bb01b09bfc7409f4d70d264b3329ad4`. +The [nine-item review record](2026-11-02-aggregate-select-review.md) supersedes +its coverage claims and private verification commands. Its combined counts do +not establish generated-only reach. The TypeScript hash below identifies an +empty output file, not a standalone proof of the command or its exit status. +The original source and logs remain historical evidence, not final-head evidence. + +## Binding and diagnosis + +Base: `a3c4d0676621cc7df4d242bdfe31815509add734`, the exact main commit +of Release run 41484825362. Job 125412837396 failed the random +`aggregate.select-shapes` campaign at its final reach assertion. Both string +ordering states had count zero. The job reports 14,748 passing tests, one +failed test, 196 files and no type errors. No production comparison mismatch +was reported. The coverage map had already recorded this sampling gap. + +The ordered-shape grammar always selects a string group key where available. +Its randomly chosen terms can all avoid that field. More random reruns cannot +establish seed-independent structural reach. This repair changes test +instrumentation only. It adds no product law or production code. + +Reviewed source: `packages/db/tests/query/aggregate-select-oracle.property.test.ts`, +SHA256 `bf90c7266c23765220384d5a027de1810bb01b09bfc7409f4d70d264b3329ad4`. This record binds that source hash to the results below; +the commit containing this record supplies its immutable Git binding. + +## Law, reach and evidence + +**Law and authority:** The existing aggregate owner, the query guide's +“Ordering by SELECT Fields” and “Multiple Column Ordering”, and the live-query +architecture's aggregate-select law remain unchanged. Published rows match +independent aggregate recomputation. Explicit string order uses `en-US` +collation. ORC-015 requires the declared premises to be reached. + +**Repair:** Twelve structural histories use the same `runHistory`, independent +model and published-row comparison as the broad grammar. Both campaigns run +those histories first, in addition to their unchanged 160 generated histories. +`createOracleReach(..., { fixedHistories: 12 })` requires each of the 21 +states in the structural set itself. The minimum remains one. The fixed seed +remains 2094; the random campaign retains its fresh seed. A selected seed/path +runs only the requested replay through the shared replay interface. + +**RED:** The authoritative main CI failure above establishes the old missing +reach. Its seed was not captured because the failure occurred after the +property completed. The first local structural draft also failed correctly: +its count-ordered pair groups all tied, so three declared ordering states had +no structural witness. Replacing that case's count with a sum establishes +distinct blocks. This intermediate failure was a reach failure, not a runtime +defect or final-source evidence. + +**GREEN:** With unchanged production, the final source passes all six tests: +twelve structural plus 160 generated histories in each campaign, and four +existing model replays. Focused TypeScript, formatting, whitespace and `pnpm lint:changed` pass. ESLint reports zero errors +and three existing `no-shadow` warnings on unchanged declarations. Cached +local Vitest is 3.2.4; CI uses 4.1.12. No installation or lockfile change ran. + +**Calibration:** Removing the final structural string history makes the fixed +campaign fail its fixed-reach assertion for both string states and three-key +groups. The broad generator still runs. This rejects random reach as a +substitute for structural coverage. Changing the driver's requested collation +to `lexical` fails `initial: top-level rows in orderBy position 0`, with +`A` before `a` instead of the model's locale order. Seed 2094/path 11 repeats +that assertion failure in one test, with the other five tests standing aside. +Restoring the source makes the same replay pass. These are assertion failures +at the intended comparison, not timeouts or setup failures. All temporary +mutants were restored; production sources were never edited. + +Normal command, from the isolated checkout: + +```sh +TANSTACK_DB_ORACLE_STATISTICS=1 pnpm_config_verify_deps_before_run=false node node_modules/vitest/vitest.mjs run --config /private/tmp/callback-metadata/aggregate-select-reach/vitest.config.mjs +``` + +Replay adds `TANSTACK_DB_ORACLE_PROPERTY=aggregate.select-shapes`, +`TANSTACK_DB_ORACLE_SEED=2094` and `TANSTACK_DB_ORACLE_PATH=11`. +The config uses this checkout's DB, db-ivm and store source, jsdom, the DB +setup file and one worker, without coverage or broad typecheck runs. + +| Declared state | Fixed campaign histories | Random campaign histories | +| --- | ---: | ---: | +| bare include: field | 12 | 17 | +| bare include: nested | 13 | 8 | +| bare include: case | 10 | 12 | +| toArray include in a conditional branch | 12 | 12 | +| bare group keys without an aggregate | 18 | 30 | +| bare group keys beside an aggregate | 64 | 66 | +| include() wrap | 24 | 34 | +| materialize() wrap | 23 | 29 | +| grouping stripped alone: grouped | 30 | 26 | +| grouping stripped alone: one group | 10 | 8 | +| grouping stripped alone: HAVING aggregate | 28 | 25 | +| include runtime stripped | 14 | 14 | +| orderBy aggregate stripped beside a select aggregate | 33 | 21 | +| an aggregate orders the published rows | 25 | 16 | +| a number select field orders the published rows | 12 | 11 | +| a string select field orders the published rows | 3 | 6 | +| a string field orders A against a lowercase key | 3 | 5 | +| two orderBy terms order the rows | 10 | 10 | +| an include query orders its rows | 32 | 27 | +| two group keys publish several groups | 8 | 7 | +| three group keys publish several groups | 7 | 2 | + +Each state also occurred in the structural set, as its separate enforced +fixed-reach assertion proves. Counts include the structural set and generated +histories; they are not generated-only counts. + +## Oracle review + +This is a self-review, not a fresh external review. + +| Requirement | Outcome | +| --- | --- | +| ORC-001 | Existing authority and domain retained; this repair closes sampling reach only. | +| ORC-002 | Independent model and locale comparator unchanged. | +| ORC-003 | Opening grammar prose and structural-history prose explain the additional premises. | +| ORC-004 | Existing broad grammar unchanged. Structural cases reconstruct the previously legal string witness. Removing it fails reach. The adjacent `upper` grouping collapses `a`/`A`; `key`, `pair` and `triple` retain raw keys. Source fields outside group keys and aggregates remain rejected. | +| ORC-005 | Real Collection queries and initial/after-step public comparisons retained. No missing-source-input hypothesis was identified; the driver preloads all initial rows. | +| ORC-006 | Code-unit collation injection fails the published-row comparison and its direct replay. | +| ORC-007 | Same examples, generator, driver, comparison and 160 generated runs in fixed and fresh campaigns. Direct replay runs one requested history. | +| ORC-008 | No reference-model state changed. | +| ORC-009 | No new subsystem concept or model abstraction. `fixed` labels a test history's structural provenance. | +| ORC-010 | Existing cleanup unchanged. Fast-check captures the structural counterexample and replay path. | +| ORC-011 | No new shared-model semantic hypothesis; ordering authority and independent comparator unchanged. | +| ORC-012 | This record binds all outcomes to the source hash and preserves intermediate failures separately. | +| ORC-013 | `a` versus `A` distinguishes locale from code-unit order at initial publication. | +| ORC-014 | Claim stays at the existing controlled Collection/jsdom boundary. No new provider or framework claim. | +| ORC-015 | All 21 minima retained; each is now additionally required in the structural set. | +| ORC-016 | Structural inputs replay the shared model and comparison without separate expected values; no unit test added. | + +**Limits:** This establishes stable reach of existing declared states. It does +not prove all possible shapes, all mutations or all comparator behaviors. +The original owner still excludes raw-expression ordering and boolean, Date, +NaN and object order fields. The complete structural histories have no mutations; the +unchanged generated histories vary updates, deletes, routes, shadowing, +operator removal and shape expressions. Direction and multi-term sensitivity +outside this collation calibration retain the owner's historical evidence. +No full repository test/build, publication, or release PR merge ran. + +## Local evidence hashes + +Logs are under `/private/tmp/callback-metadata/aggregate-select-reach`. + +| Log | SHA256 | +| --- | --- | +| `first-tests.log` | `056deb18796f2d4d74dd777c73bd194503d1856c37caeacd30d155e118c4b51d` | +| `final-tests.log` | `5622b1e0564c9922618001287d509c0a10107a62a1e55eefc6193168cc5a4be6` | +| `missing-structural-string.log` | `533a83212c25eb3a45a874e2351fba44ef64e5c71cd15c8383f9440cb74a7e3c` | +| `code-unit-mutant.log` | `4b458e6abc7e98bada1ff19cb8dacf8265130607efb752e3f80519b1cbcf132a` | +| `code-unit-mutant-replay.log` | `2d17f5f8e4409e743ec16783ba21c924c4846c13def96cfa44ee2cb03b8f1a1f` | +| `repaired-replay.log` | `ac5d720b4e6a027c2ad3c1fa798b5667e44259248d0563edde2990bbb27dd2c5` | +| `types.log` | `e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855` | +| `eslint.log` | `0a30e682ec11fc930d253804af5bc4df8de96e0972b64b856500704128b1f0cf` | +| `lint-changed.log` | `89d7da780deb04850db632f478bc4155d23659849c0e5bec6d114c74dc9d5168` | diff --git a/docs/contributing/oracle-reviews/2026-11-02-aggregate-select-review.md b/docs/contributing/oracle-reviews/2026-11-02-aggregate-select-review.md new file mode 100644 index 0000000000..1d157065cf --- /dev/null +++ b/docs/contributing/oracle-reviews/2026-11-02-aggregate-select-review.md @@ -0,0 +1,217 @@ +# Aggregate select review, 2026-11-02 + +## Contract and implementation state + +The nine findings concern test integrity, diagnostics, documentation, and +maintainability. They identify no production defect or new product law. +The reviewer read the diff, did not run the oracle, and verified the original +source hash. This follow-up supplies executed experiments. + +Reviewed remote head: `df5353866bbc99aed92e0ae3361901f2007d32c2`. +Current-main integration: `8cb66ee011cb7c484dbdf5cd81c14d48bf09491c`, with parents +`df5353866bbc99aed92e0ae3361901f2007d32c2` and +`9be51e037f6c092bc9625f9b5b4b7e23eb4fa5fe`. +The aggregate source SHA256 for the results below is `630602aebe41bfd39804c482a2c0a7deb60194a34d865ccd467de4f824fc52c7`. +The commit containing this record binds the source, configs, and shared helpers. +Production, dependencies, and the lockfile are unchanged relative to current main. + +**Law and authority:** The aggregate owner and the query guide retain their +existing law: compilation follows the select/operator rules, and published +rows equal independent group recomputation, including the documented locale +order. ORC-015 requires declared premises to be reached. ORC-007 requires parity +between fixed and fresh campaigns. It also requires direct replay. A fixed +example cannot supply evidence about a generated history. Diagnostic seed +selection is not execution proof; guarded replay witnesses retain that role. + +**Repair:** Each campaign has a structural reach for the 21 initial premises +and a separate generated reach for those same states plus seven successful +mutation/route comparison checkpoints. The 12 structural inputs are complete +histories with no mutation steps, shadowed alias, or union. They are not +prefixes of generated histories. The generated budget remains 160, minima +remain one, and fixed seed 2094 remains. Named history fields replace the +10-slot tuple; campaign callbacks determine structural provenance. + +Shared parameter helpers choose and report a fresh seed when no seed is given. +They preserve explicit seeds, path selection, example accounting, replay +reporters, and one-history path replays. The campaign runner reports seed and +counts even on failure. Aggregate statistics run in `finally`, use `=== '1'`, +and clear the current reach after failure. A seed-only replay reruns a reach +campaign; a reach failure after successful comparisons has no shrink path. + +## Experiments and inference + +Before repairs, all three hostile generator edits passed the fixed campaign: +ordered-lane weight zero, no string terms from the ordered lane, and no runtime +stripping from the generated grammar. The structural histories concealed every +missing premise. The same edits failed an independent generated-reach probe, +then failed the repaired branch at the generated reach assertion. Removing +steps, shadowing, or union from the generated grammar also fails the new +checkpoint assertions. These are coverage failures, not production mismatches. + +Changing the structural triple input to a single key fails structural reach +even though generated histories still reach triple groups. Selecting lexical +collation in the driver fails the public row-order comparison. Seed 2094/path +11 reproduces that mismatch directly; restoring the driver makes that replay +pass (one test runs, five stand aside). + +The shared declared-reach model first failed against unchanged helpers because +a failing campaign omitted counts and fresh options omitted a chosen seed. +It now checks generated run kinds, exact budgets/examples, replay parameters, +seed diagnostics, and counts on failing campaigns. Removing either fresh-seed +propagation or its diagnostic line fails that owner. Existing replay-reporter +checks retain native failure formatting and cause behavior. + +A fresh reach-only failure selected seed `373827046`. Repeating that +seed reproduced the same generated counts and the same missing-runtime state. +With statistics `0`, aggregate counts do not print. With statistics `1`, the +injected reach failure retains both structural and generated counts. + +GREEN: 14 focused tests pass across the aggregate owner, declared-reach owner, +campaign helper, and reach helper. Both aggregate campaigns run, plus all four +existing model replays. Another 38 reporting/manifest checks pass; 44 unrelated +child-process replay checks stand aside under an explicit test selector. +Focused TypeScript exits zero with `--listFiles`, which lists the intended +owners and helpers. Local cached Vitest is 3.2.4; repository CI uses 4.1.12. +No install, override, dependency edit, or production edit ran. + +**Sibling S01:** A fresh seed is not an explicit replay. Auditing `.seed` +consumers found three sites that used seed presence to select replay behavior: +structural-identity fixed campaigns, layered-publication campaigns, and the +serializer shrink/replay calibration. They now read explicit replay selection +from `readOracleRunConfig`. The calibration keeps seed 20260914 in ordinary +runs. A control-plane probe reproduced the omitted-campaign decision before +repair. All 131 tests in the three affected owners then pass, including fixed +and fresh publication campaigns and the serializer replay calibration. This +preserves the existing contracts, generators, comparisons, and budgets. + +## Reproduce from a checkout + +These commands use committed configs and resolve the checkout's own source. +They require the repository dependencies already installed. + +```sh +TANSTACK_DB_ORACLE_STATISTICS=1 pnpm exec vitest run --config packages/db/vitest.aggregate.config.mjs tests/query/aggregate-select-oracle.property.test.ts tests/declared-reach-oracle.property.test.ts tests/oracle-campaign.test.ts tests/oracle-reach.test.ts +pnpm exec vitest run --config packages/db/vitest.aggregate.config.mjs tests/oracle-replay.test.ts -t 'oracle replay reporting|named oracle replay manifest' +pnpm exec tsc --project packages/db/tsconfig.aggregate.json --noEmit --listFiles +pnpm exec vitest run --config scripts/oracle-diagnostics.vitest.config.mjs +TANSTACK_DB_ORACLE_PROPERTY=aggregate.select-shapes TANSTACK_DB_ORACLE_SEED=2094 TANSTACK_DB_ORACLE_PATH=11 pnpm exec vitest run --config packages/db/vitest.aggregate.config.mjs tests/query/aggregate-select-oracle.property.test.ts +pnpm lint:changed +``` + +The first command runs the default fixed and fresh campaigns. To replay a +reach miss, set `TANSTACK_DB_ORACLE_SEED` to its `ORACLE_SEED` value and select +the random test with `-t 'random or replayed'`. Do not add a path for a whole +campaign reach miss. The fifth command above demonstrates a path replay, not +a whole-campaign reach check. An empty successful TypeScript output is normal; +record the command and exit status instead of inferring success from its hash. + +## Separate reach counts + +Final local fresh aggregate seed: `261424648`. The structural columns +count only fixed initial histories. Generated columns exclude them. + +| State | Fixed structural | Fixed generated | Fresh structural | Fresh generated | +| --- | ---: | ---: | ---: | ---: | +| bare include: field | 1 | 11 | 1 | 6 | +| bare include: nested | 1 | 12 | 1 | 12 | +| bare include: case | 1 | 9 | 1 | 8 | +| toArray include in a conditional branch | 1 | 11 | 1 | 9 | +| bare group keys without an aggregate | 1 | 17 | 1 | 25 | +| bare group keys beside an aggregate | 1 | 63 | 1 | 70 | +| include() wrap | 1 | 23 | 1 | 25 | +| materialize() wrap | 1 | 22 | 1 | 23 | +| grouping stripped alone: grouped | 1 | 29 | 1 | 20 | +| grouping stripped alone: one group | 1 | 9 | 1 | 9 | +| grouping stripped alone: HAVING aggregate | 1 | 27 | 1 | 22 | +| include runtime stripped | 1 | 13 | 1 | 13 | +| orderBy aggregate stripped beside a select aggregate | 1 | 32 | 1 | 33 | +| an aggregate orders the published rows | 1 | 24 | 1 | 21 | +| a number select field orders the published rows | 1 | 11 | 1 | 9 | +| a string select field orders the published rows | 1 | 2 | 1 | 6 | +| a string field orders A against a lowercase key | 1 | 2 | 1 | 4 | +| two orderBy terms order the rows | 1 | 9 | 1 | 10 | +| an include query orders its rows | 2 | 30 | 2 | 28 | +| two group keys publish several groups | 1 | 7 | 1 | 12 | +| three group keys publish several groups | 1 | 6 | 1 | 7 | +| published after a parent update | — | 33 | — | 36 | +| published after a source insert | — | 32 | — | 31 | +| published after a source update | — | 32 | — | 38 | +| published after a source delete | — | 30 | — | 34 | +| published after a source route move | — | 10 | — | 13 | +| a shadowed child alias publishes rows | — | 15 | — | 11 | +| a union source publishes rows | — | 32 | — | 29 | + +## Lossless finding accounting + +| ID | Technical verdict and severity | Action and durable value | +| --- | --- | --- | +| R01 | Confirmed test-integrity flaw, high. Structural inputs masked all generated reach. | Fixed: independent generated minima and hostile lane/string/runtime ablations. | +| R02 | Confirmed documentation overclaim, medium. Initial coverage does not establish mutations or routes. | Fixed: exact coverage-map limit and seven generated-only comparison checkpoints. They do not claim a full state-by-mutation cross-product. | +| R03 | Confirmed prose error, low. These inputs have neither suffix nor recorded replay origin. | Fixed: complete structural histories and their exact initial scope. | +| R04 | Confirmed reproducibility gap, medium. An empty TypeScript log alone does not prove a failed command. Version and integration caveats are accurate. | Fixed: repository-relative configs, current integration verification, command/exit/listed-file evidence, and explicit historical evidence. Final CI on Vitest 4.1.12 remains a separate readiness gate. | +| R05 | Confirmed shared diagnostic gap, medium. A final reach failure can follow fast-check success without a reported seed. | Fixed: shared chosen-seed diagnostics, seed/count relation checks, and executed seed-only replay. Direct fast-check callers outside these helpers retain their own diagnostic responsibility. | +| R06 | Confirmed maintainability issue, low. The manual count was correct but duplicated shared policy. | Fixed: `oracleRandomParameters` accounts for examples. The literal budget/seed object remains visible to the existing budget auditor. | +| R07 | Confirmed diagnostic flaw, medium. The post-assert statistics never ran on a reach miss. | Fixed: one aggregate runner prints in `finally`; the shared campaign owner checks failing-campaign counts. | +| R08 | Confirmed consistency error, low. The string `0` enabled aggregate statistics. | Fixed: exact `1` gate, checked with both values. Seed-selection diagnostics are separate from optional statistics. | +| R09 | Confirmed maintainability issue, low. Fixedness was an interchangeable boolean in the generated tuple. | Fixed: named history fields and callback-owned provenance; no dummy constant remains. | + +All nine source-order items have dispositions: 9 fixed now, 0 omitted, 0 +confirmed open, 0 deferred, and 0 unresolved law decisions. R04 contains an +important qualification, not evidence that the historical TypeScript run failed. +Sibling S01 is also fixed. No production bug is claimed. + +## Reviewer assessment and limits + +Hire recommendation: **yes** for careful code review. The review correctly +prioritized a false green, connected it to lost replay diagnostics, separated +production defects from test integrity, and proposed idiomatic shared repairs. +Its evidence criticism needed the TypeScript qualification. It supplied no +executed counterexample; the experiments above establish the behavioral claims. + +This is a self-review of the repairs. Earlier CodeRabbit coverage of the +original three files does not establish external review of the shared helper +changes. Readiness remains unproven until final pushed-head CI completes. +The controlled boundary is real Collection/live-query code in jsdom. No new +provider or framework-browser coverage is claimed. The structural guarantee +covers initial compilation/publication only. Generated coverage is sampled, +not every law after every mutation, route, or shape. A missed generated state +fails; it is never silently supplied by structural coverage. All existing +shape/type exclusions remain. No full repository campaign, release merge, +publisher, or publication verification ran for this review. + +## CI configuration expectation follow-up + +At head `a0e323a7750a24621a2668a3c8f2a761865872d2`, CI run `41521949066`, +job `125539942181`, found one stale configuration assertion in `utils.test.ts`. +It expected ordinary options to omit the seed, although the reviewed helper +change now selects one. The focused local reproduction failed at that same +assertion: one failed test, 15 passed, and 116 stood aside under the selector. + +The assertion now expects a numeric seed. Exact object equality still rejects +an unexpected path or reporter. The neighboring assertions retain the explicit +seed for another property and the one-run shrink path for its named property. +All 132 tests in `utils.test.ts` pass after this correction. The shared helpers, +oracle models, generated histories, and production code are unchanged from +`a0e323a`. Earlier oracle and mutant results remain source-identical evidence; +they were not rerun for this assertion and wording correction. New-head CI +remains required. + +CodeRabbit completed its review of `a0e323a` with 15 selected files and two +similar files skipped. Its one additional comment requested clearer ORC-007 +wording above. That wording is corrected. This review did not supply executed +oracle evidence, and its coverage does not establish review of a later head. + + +## Native adapter CI expectation follow-up + +The `ee13953af78832537b63f16649bba17bf4ced01d` PR workflow, attempt 1, failed the `Test (sqlite-adapters)` job `125548643697`. The ordinary test `op-sqlite-driver.test.ts:328` expected only `{ numRuns: 50 }`. The approved fresh-seed contract also returns a numeric `seed`. This is another stale option assertion, not an observed driver mismatch or reach failure. + +The unchanged source reproduces that assertion failure locally: one test fails and 92 tests are filtered. An initial dependency setup failure is retained separately. The repaired assertion compares the complete object with `{ numRuns: runs, seed: expect.any(Number) }`. It still rejects an unexpected path or reporter. The test name now describes fresh seed selection and the unchanged campaign budget. The neighboring explicit-seed and named-path assertions remain unchanged. + +All 93 ordinary tests in `op-sqlite-driver.test.ts` pass after the edit. The local run uses cached Vitest 3.2.4, Node SQLite fixtures, and the existing mobile setup file. CI uses Vitest 4.1.12. No mobile runtime or new oracle campaign ran. The alias oracle, shared seed helpers, models, generators, production, dependencies, and lockfile remain unchanged from `ee13953`. The earlier bounded oracle evidence remains source-bound historical evidence. + +A repository search of direct option assertions finds no other expectation that omits the selected seed. This search does not prove every indirect consumer contract. The existing shared diagnostic oracle and its earlier calibration remain unchanged. This repair updates an ordinary configuration expectation under the already accepted seed contract. + +Local reproduction uses `pnpm exec vitest run --config /private/tmp/callback-metadata/2331-native-ci-expectation-20261102/native.vitest.config.mjs`. The normal repository package command remains `pnpm --filter @tanstack/react-native-db-sqlite-persistence test`. The scratch configuration selects the ordinary file and existing source aliases with one worker. It does not establish complete package-suite or mobile-runtime coverage. + +Full command, exit, source, and log bindings remain under `/private/tmp/callback-metadata/2331-native-ci-expectation-20261102`. The complete authoritative job log has SHA-256 `ba29458721937e64a447133ba4bdc0c66b74ad631763d5ea866bdccb5930f71f`. The workflow was still active when its run object was fetched. The failed job is decisive, but its status is not a final workflow conclusion. No CI retry ran. Final pushed-head CI remains required. diff --git a/packages/db-ivm/tests/structural-identity-oracle.property.test.ts b/packages/db-ivm/tests/structural-identity-oracle.property.test.ts index c901793348..e82ee18118 100644 --- a/packages/db-ivm/tests/structural-identity-oracle.property.test.ts +++ b/packages/db-ivm/tests/structural-identity-oracle.property.test.ts @@ -63,7 +63,7 @@ type Spec = const SYMBOL = Symbol(`s`) const ENUMERABLE_SYMBOL = Symbol(`enumerable`) -const { replayProperty } = readOracleRunConfig() +const { replayProperty, replaySeed } = readOracleRunConfig() const LEAVES: ReadonlyArray = [ 0, -0, @@ -437,7 +437,7 @@ describe(`structuralIdentity`, () => { 2000, `structural-identity.equal-representations`, ) - if (options.seed === undefined) + if (replaySeed === undefined) await fc.assert(property, { numRuns: options.numRuns, seed: 20261014 }) await fc.assert(property, options) }) @@ -597,7 +597,7 @@ describe(`structuralIdentity`, () => { 2000, `structural-identity.independent-values`, ) - if (options.seed === undefined) + if (replaySeed === undefined) await fc.assert(property, { numRuns: options.numRuns, seed: 1158727847 }) await fc.assert(property, options) }) diff --git a/packages/db/tests/declared-reach-oracle.property.test.ts b/packages/db/tests/declared-reach-oracle.property.test.ts index 939195b898..6017b41a31 100644 --- a/packages/db/tests/declared-reach-oracle.property.test.ts +++ b/packages/db/tests/declared-reach-oracle.property.test.ts @@ -34,8 +34,10 @@ * `runOracleCampaign` runs a fixed-seed and a random campaign, each with its * own reach and a minimum of one. A seed-only run of its replay prefix runs one * campaign with that seed and checks its reach; a seed and path run one - * recorded history and check none. Each campaign that passes reports its - * counts on one `ORACLE_REACH` line. An invalid prefix environment throws. + * recorded history and check none. Every started campaign reports its seed and + * counts on one `ORACLE_REACH` line, including a failing campaign. Fresh seed + * selection is diagnostic data, not proof that a comparison ran. An invalid + * prefix environment throws. * * The model below states these rules over a generated ledger: a run kind, an * optional `replaying` override, the options, the states reached before the @@ -373,6 +375,7 @@ type CampaignLine = { property: string campaign: string minimum: number | null + seed: number counts: Record } type CampaignObservation = { @@ -400,8 +403,8 @@ function campaignLedger(ledger: Ledger): Ledger { // The campaign model: two campaigns without a prefix seed, one replay // campaign with one. Each campaign has a minimum of one and no fixed -// histories, checks reach unless a path replays it, and reports its counts -// when it passes. The first failing campaign ends the run. +// histories, checks reach unless a path replays it, and reports its seed and counts +// even when reach fails. The first failing campaign ends the run. function expectedCampaign(ledger: Ledger): CampaignObservation { if (ledger.campaignFault !== undefined) { return { parameters: [], lines: [], failure: `invalid environment` } @@ -413,7 +416,7 @@ function expectedCampaign(ledger: Ledger): CampaignObservation { run === `ordinary` ? [ [`fixed`, { seed: ledger.fixedSeed }], - [`random`, {}], + [`random`, { seed: expect.any(Number) }], ] : [ [ @@ -433,13 +436,14 @@ function expectedCampaign(ledger: Ledger): CampaignObservation { fixedHistories: 0, replaying: run === `path`, }) - if (failure !== undefined) return { parameters, lines, failure } lines.push({ property: campaignPrefix, campaign, minimum: run === `path` ? null : 1, + seed: campaignParameters.seed!, counts: Object.fromEntries(counts), }) + if (failure !== undefined) return { parameters, lines, failure } } return { parameters, lines, failure: undefined } } @@ -458,13 +462,21 @@ function observeCampaign(ledger: Ledger): Promise { }> = [] // The runner reports each campaign's counts on the console. const info = vi.spyOn(console, `info`).mockImplementation(() => {}) - const lines = () => - info.mock.calls.map( - ([line]) => - JSON.parse( - String(line).replace(/^ORACLE_REACH /, ``), - ) as CampaignLine, - ) + const lines = () => { + const observations = info.mock.calls + .filter(([line]) => String(line).startsWith(`ORACLE_REACH `)) + .map( + ([line]) => + JSON.parse( + String(line).replace(/^ORACLE_REACH /, ``), + ) as CampaignLine, + ) + for (const [index, line] of observations.entries()) + expect(line.seed, `reported campaign seed`).toBe( + parameters[index]?.seed, + ) + return observations + } try { await runOracleCampaign( campaignPrefix, @@ -518,7 +530,7 @@ function expectedRunParameters( examples: ReadonlyArray, ) { const numRuns = run === `path` ? 1 : baseRuns + examples.length - if (run === `ordinary`) return { numRuns, examples } + if (run === `ordinary`) return { numRuns, seed: expect.any(Number), examples } if (run === `path`) return { numRuns, seed, path, examples } return { numRuns, seed, examples } } @@ -558,23 +570,43 @@ function checkRunParameters( path: parameters.path, examples: parameters.examples, }) + const observe = (select: () => T): T => { + const info = vi.spyOn(console, `info`).mockImplementation(() => {}) + try { + const parameters = select() + expect( + Number.isSafeInteger(parameters.seed), + `selected seed is replayable`, + ).toBe(true) + expect( + info.mock.calls.map(([line]) => + JSON.parse(String(line).replace(/^ORACLE_SEED /, ``)), + ), + ).toEqual([{ property, seed: parameters.seed }]) + return parameters + } finally { + info.mockRestore() + } + } const replayed = run === `path` ? property : otherProperty // toEqual treats an undefined key as absent. expect( project( - oracleRandomParameters( - baseRuns, - run === `ordinary` - ? undefined - : run === `seed` - ? { - replaySeed: 1, - replayPath: undefined, - replayProperty: undefined, - } - : { replaySeed: 1, replayPath: `0`, replayProperty: replayed }, - property, - examples, + observe(() => + oracleRandomParameters( + baseRuns, + run === `ordinary` + ? undefined + : run === `seed` + ? { + replaySeed: 1, + replayPath: undefined, + replayProperty: undefined, + } + : { replaySeed: 1, replayPath: `0`, replayProperty: replayed }, + property, + examples, + ), ), ), `run parameters`, @@ -582,7 +614,9 @@ function checkRunParameters( expectedRunParameters({ run, seed: 1, path: `0` }, baseRuns, examples), ) expect( - project(oraclePropertyOptions(baseRuns, property, { examples })), + project( + observe(() => oraclePropertyOptions(baseRuns, property, { examples })), + ), `property options under this process`, ).toEqual( expectedRunParameters( diff --git a/packages/db/tests/oracle-campaign.test.ts b/packages/db/tests/oracle-campaign.test.ts index 1a3aa01dfa..90ca4b6b8a 100644 --- a/packages/db/tests/oracle-campaign.test.ts +++ b/packages/db/tests/oracle-campaign.test.ts @@ -10,10 +10,10 @@ it(`does not let fixed campaign reach conceal an unreached random state`, async vi.spyOn(console, `info`).mockImplementation(() => {}) let campaigns = 0 await expect( - runOracleCampaign(`TEST_ORACLE`, [`required`], (reach, parameters) => { + runOracleCampaign(`TEST_ORACLE`, [`required`], (reach) => { campaigns++ reach.beginHistory() - if (parameters.seed !== undefined) reach.reach(`required`) + if (campaigns === 1) reach.reach(`required`) }), ).rejects.toThrow(/required \(0\)/) expect(campaigns).toBe(2) diff --git a/packages/db/tests/oracle-campaign.ts b/packages/db/tests/oracle-campaign.ts index d99f4fb480..c5bfafa96e 100644 --- a/packages/db/tests/oracle-campaign.ts +++ b/packages/db/tests/oracle-campaign.ts @@ -1,9 +1,11 @@ +import { oracleCampaignSeed } from './oracle-config' import { createOracleReach } from './oracle-reach' /** * Run the same oracle with separate fixed/random reach ledgers. The callback * owns its property, grammar, recorder, comparison, and run budget; only seed - * selection changes between the two calls. Counts are histories, not events: + * selection changes between the two calls. Every started campaign reports its + * seed and counts even when the callback or reach assertion fails. Counts are histories, not events: * call beginHistory once and reach only after observing the named checkpoint. * A prefix SEED replay runs one campaign at the callback's full budget, and * checks its reach. A SEED/PATH replay runs one recorded history and checks @@ -54,15 +56,23 @@ export async function runOracleCampaign( const reach = createOracleReach(prefix, states, { replaying: path !== undefined, }) - await run(reach, campaign.parameters) - reach.assertReached() - console.info( - `ORACLE_REACH ${JSON.stringify({ - property: prefix, - campaign: campaign.name, - minimum: path === undefined ? 1 : null, - counts: Object.fromEntries(reach.counts()), - })}`, - ) + const parameters = { + ...campaign.parameters, + seed: oracleCampaignSeed(campaign.parameters.seed, prefix), + } + try { + await run(reach, parameters) + reach.assertReached() + } finally { + console.info( + `ORACLE_REACH ${JSON.stringify({ + property: prefix, + campaign: campaign.name, + seed: parameters.seed, + minimum: path === undefined ? 1 : null, + counts: Object.fromEntries(reach.counts()), + })}`, + ) + } } } diff --git a/packages/db/tests/oracle-config.ts b/packages/db/tests/oracle-config.ts index 4117dbabe9..08158d7592 100644 --- a/packages/db/tests/oracle-config.ts +++ b/packages/db/tests/oracle-config.ts @@ -726,6 +726,23 @@ export function readOracleRunConfig( return { multiplier, replaySeed, replayPath, replayProperty } } +/** + * Select a fresh campaign seed unless the caller supplied one. Report the + * selection before fast-check runs: a later reach failure is outside its + * reporter and must still be replayable. This line records selection, not + * execution; guarded replay witnesses remain the execution evidence. + */ +export function oracleCampaignSeed( + seed: number | undefined, + property: string | undefined, +): number { + const selected = seed ?? Math.floor(Math.random() * 0x100000000) - 0x80000000 + console.info( + `ORACLE_SEED ${JSON.stringify({ property: property ?? null, seed: selected })}`, + ) + return selected +} + type OracleRunParameters = { numRuns: number seed?: number @@ -770,9 +787,10 @@ function randomParameters( replayPath: undefined, replayProperty: undefined, } + const selectedSeed = oracleCampaignSeed(replaySeed, property) const parameters: OracleRunParameters = replaySeed === undefined - ? { numRuns } + ? { numRuns, seed: selectedSeed } : { numRuns, seed: replaySeed, diff --git a/packages/db/tests/oracle-replay.test.ts b/packages/db/tests/oracle-replay.test.ts index d8a67366e9..a188f77ab7 100644 --- a/packages/db/tests/oracle-replay.test.ts +++ b/packages/db/tests/oracle-replay.test.ts @@ -329,9 +329,12 @@ describe(`oracle replay reporting`, () => { directory = undefined }) - it(`leaves ordinary seeded and random options unchanged`, () => { + it(`preserves explicit seeds and supplies a fresh seed for ordinary runs`, () => { vi.stubEnv(`TANSTACK_DB_ORACLE_REPLAY_WITNESS`, undefined) - expect(oracleRandomParameters(5, undefined)).toEqual({ numRuns: 5 }) + expect(oracleRandomParameters(5, undefined)).toEqual({ + numRuns: 5, + seed: expect.any(Number), + }) expect(oracleRandomParameters(5, 42)).toEqual({ numRuns: 5, seed: 42 }) expect(oracleReplayReporter(`oracle-replay.calibration`, 42, `0`)).toEqual( {}, diff --git a/packages/db/tests/query/aggregate-select-oracle.property.test.ts b/packages/db/tests/query/aggregate-select-oracle.property.test.ts index 83f6a50131..cd25d6a81b 100644 --- a/packages/db/tests/query/aggregate-select-oracle.property.test.ts +++ b/packages/db/tests/query/aggregate-select-oracle.property.test.ts @@ -36,7 +36,12 @@ import { NonAggregateExpressionNotInGroupByError, } from '../../src/errors.js' import { mockSyncCollectionOptions, stripVirtualProps } from '../utils.js' -import { oraclePropertyOptions, oracleRuns } from '../oracle-config.js' +import { + oraclePropertyOptions, + oracleRandomParameters, + oracleReplayTest, + oracleRuns, +} from '../oracle-config.js' import { createOracleReach } from '../oracle-reach.js' import { declareOracle } from '../oracle-declaration.js' @@ -138,8 +143,8 @@ import { declareOracle } from '../oracle-declaration.js' * top-level query a select of `count()` alone, with an aggregate HAVING when * the generated shape has one. One history in four is such a forced * history: it strips the grouping compiler everywhere or from orderBy only, - * with no grouping, one key, or two keys, in turn, so every campaign reaches - * each path whatever its seed. A model error set of exactly + * with no grouping, one key, or two keys, to sample each compiler path. A + * model error set of exactly * `groupingRequired` then witnesses each path, grouped, one implicit group, * and HAVING, where no other error can hide the missing compiler. The * property declares the states these laws need with `createOracleReach` and @@ -162,7 +167,13 @@ import { declareOracle } from '../oracle-declaration.js' * with an orderBy rarely compile, so one history in four takes ordered * shapes from a grammar whose shapes compile. The source starts with a row * whose key `A` sorts between `a` and `b`, where locale and code-unit order - * differ. + * differ. Each campaign first runs twelve complete structural histories through + * this same model, driver, and comparison. They guarantee the 21 initial + * premises, including string ordering, independently of the seed, with no + * mutations, shadowed alias, or union. A separate reach requires the 160 + * generated histories themselves to enter those states and the mutation and + * route checkpoints below. Their coverage is sampled: a miss fails with the + * selected campaign seed and counts, rather than borrowing structural reach. * * Limits: one parent field (`x`) and one source field (`v`); groups by one, * two, or three keys; numbers and string keys only. A boolean, Date, `NaN`, @@ -1120,17 +1131,31 @@ function withoutOrderByGrouping(ir: any): any { } } -async function runHistory( - top: Shape, - nested: Shape, - grouping: Grouping, - shadow: boolean, - steps: ReadonlyArray, +// History provenance belongs to the campaign, not to the generated input. +// Named options keep routing and operator-removal choices distinct. +type History = { + top: Shape + nested: Shape + grouping: Grouping + shadow?: boolean + steps?: ReadonlyArray + union?: boolean + bareKeys?: boolean + stripped?: Stripped + alone?: boolean +} + +async function runHistory({ + top, + nested, + grouping, + shadow = false, + steps = [], union = false, bareKeys = false, - stripped: Stripped = `none`, + stripped = `none`, alone = false, -): Promise { +}: History): Promise { const grouped = grouping !== `none` // Most generated shapes throw on their own; a runtime-stripped history // needs an include that only the missing runtime can fail. @@ -1152,7 +1177,6 @@ async function runHistory( first?.by === `agg` ? first : { by: `agg`, fn: `sum`, dir: `asc` } top = { fields: { f0: { k: `agg`, fn: term.fn } }, orderBy: [term] } } - reach?.beginHistory() if (stripped === `runtime`) reach?.reach(`include runtime stripped`) for (const s of [top, nested]) { if (s.include?.wrap === `bare`) @@ -1458,7 +1482,11 @@ async function runHistory( } } check(`initial`) + if (shadow && includeQuery.ok) + reach?.reach(`a shadowed child alias publishes rows`) + if (union && topQuery.ok) reach?.reach(`a union source publishes rows`) for (const [index, s] of steps.entries()) { + let routeMoved = false if (s.t === `parent`) { const next = { ...issues.find((i) => i.id === s.id)!, x: s.x } issues = issues.map((i) => (i.id === s.id ? next : i)) @@ -1476,6 +1504,7 @@ async function runHistory( const target = comments[s.pick % comments.length]! if (s.t === `update`) { const row = { ...target, v: s.v, k: s.k ?? target.k } + routeMoved = row.k !== target.k comments = comments.map((c) => (c.id === target.id ? row : c)) commentSource.utils.begin() commentSource.utils.write({ type: `update`, value: row }) @@ -1489,6 +1518,17 @@ async function runHistory( } await new Promise((resolve) => setTimeout(resolve, 0)) check(`step ${index} ${JSON.stringify(s)}`) + if (topQuery.ok || includeQuery.ok) { + const checkpoint = { + parent: `published after a parent update`, + insert: `published after a source insert`, + update: `published after a source update`, + delete: `published after a source delete`, + } as const + reach?.reach(checkpoint[s.t]) + if (routeMoved && includeQuery.ok) + reach?.reach(`published after a source route move`) + } } } finally { for (const query of live) await query.cleanup() @@ -1514,26 +1554,26 @@ const historyFor = ( alone: fc.Arbitrary, shapes: typeof shape = shape, ) => - fc.tuple( - shapes(false, grouping), - shapes(true, grouping), - fc.constant(grouping), - fc.boolean(), - fc.array(step, { maxLength: 6 }), - fc.boolean(), - fc.boolean(), + fc.record({ + top: shapes(false, grouping), + nested: shapes(true, grouping), + grouping: fc.constant(grouping), + shadow: fc.boolean(), + steps: fc.array(step, { maxLength: 6 }), + union: fc.boolean(), + bareKeys: fc.boolean(), stripped, alone, - ) + }) // Half the histories are generated; three in six of those build their IR by // hand without one kind of operator code. One history in four instead strips // the grouping compiler, everywhere or from orderBy only, and takes the // forced shape, for one or several groups chosen in turn. One in four takes // ordered shapes that compile, at the top level and in the include query, -// and in one of three strips the orderBy aggregates' compiler. About 40 -// histories of each kind per campaign of 160 make each reach state certain -// in practice: a campaign misses the rarest one about once in 1,500 seeds. +// and in one of three strips the orderBy aggregates' compiler. These weights +// explore the grammar. Structural histories guarantee initial premises only; +// they never count toward the generated campaign's reach. const history = fc.oneof( { weight: 2, @@ -1584,6 +1624,63 @@ const history = fc.oneof( }, ) +/** + * Structural histories use the same model and comparison as the generated + * grammar. These complete, fixed histories have no mutations and force each + * law's initial-publication premise: sampling + * an ordered shape alone does not ensure its terms ever use a string field. + * No history has a separate expected answer. + */ +const countShape: Shape = { fields: { f0: { k: `agg`, fn: `count` } } } +const keyShape: Shape = { fields: { f0: { k: `key` } } } +const fixedHistory = ( + top: Shape, + nested: Shape = top, + grouping: Grouping = `key`, + stripped: Stripped = `none`, + bareKeys = false, +): History => ({ top, nested, grouping, stripped, bareKeys }) +const structuralHistories = [ + ...([`field`, `nested`, `case`] as const).map((at) => + fixedHistory({ ...countShape, include: { wrap: `bare`, at } }), + ), + fixedHistory({ ...countShape, include: { wrap: `toArray`, at: `case` } }), + ...([`include`, `materialize`] as const).map((wrap) => + fixedHistory({ ...countShape, include: { wrap, at: `field` } }), + ), + fixedHistory(countShape, keyShape, `key`, `none`, true), + fixedHistory(countShape, countShape, `key`, `grouping`), + fixedHistory( + { ...countShape, having: { part: { k: `agg`, fn: `count` }, lit: 2 } }, + countShape, + `none`, + `grouping`, + ), + fixedHistory(countShape, countShape, `key`, `runtime`), + fixedHistory( + { + fields: { f0: { k: `agg`, fn: `sum` } }, + orderBy: [ + { by: `agg`, fn: `sum`, dir: `desc` }, + { by: `selected`, field: `f0`, kind: `number`, dir: `asc` }, + ], + }, + undefined, + `pair`, + `orderBy`, + ), + fixedHistory( + { + ...keyShape, + orderBy: [{ by: `selected`, field: `f0`, kind: `string`, dir: `asc` }], + }, + undefined, + `triple`, + ), +] +const examples = structuralHistories.map((entry) => [entry] as [History]) +const structuralInputs = new Set(structuralHistories) + /** The states the operator laws need; each campaign must reach all of them. */ const reachStates = [ `bare include: field`, @@ -1608,153 +1705,173 @@ const reachStates = [ `two group keys publish several groups`, `three group keys publish several groups`, ] as const -let reach: - ReturnType> | undefined +// These checkpoints count only successful public comparisons from generated +// histories. Structural initial histories cannot discharge a mutation or route. +const generatedCheckpoints = [ + `published after a parent update`, + `published after a source insert`, + `published after a source update`, + `published after a source delete`, + `published after a source route move`, + `a shadowed child alias publishes rows`, + `a union source publishes rows`, +] as const +type ReachState = + (typeof reachStates)[number] | (typeof generatedCheckpoints)[number] +let reach: ReturnType> | undefined + +async function runCampaign( + campaign: string, + parameters: ReturnType>, +): Promise { + const structuralReach = createOracleReach( + `${PROPERTY} (${campaign}, structural)`, + reachStates, + { + fixedHistories: structuralHistories.length, + }, + ) + const generatedReach = createOracleReach( + `${PROPERTY} (${campaign}, generated)`, + [...reachStates, ...generatedCheckpoints], + ) + try { + await fc.assert( + fc.asyncProperty(history, async (entry) => { + const fixed = structuralInputs.has(entry) + reach = fixed ? structuralReach : generatedReach + reach.beginHistory({ fixed }) + await runHistory(entry) + }), + parameters, + ) + structuralReach.assertReached() + generatedReach.assertReached() + } finally { + // Print before unwinding a comparison or reach failure, then clear the + // campaign pointer so another test cannot enter a failed campaign's reach. + if (process.env.TANSTACK_DB_ORACLE_STATISTICS === `1`) + console.info( + JSON.stringify({ + property: PROPERTY, + campaign, + seed: parameters.seed, + structural: Object.fromEntries(structuralReach.counts()), + generated: Object.fromEntries(generatedReach.counts()), + }), + ) + reach = undefined + } +} // The check is `runHistory`: it runs each generated history, and after every // step it compares the published rows and the compile outcome with -// `modelGroup` and `modelError`. The generated properties and the pinned -// replays above reach it. +// `modelGroup` and `modelError`. Generated histories, structural examples, and +// existing model replays use this same check. declareOracle({ check: [runHistory], model: [modelGroup, modelError], + grammar: [history], }) describe(`aggregate select shapes`, () => { - it(`match the model across generated shapes (fixed campaign)`, async () => { - // A repeatable baseline. Seed 2094 is arbitrary. - reach = createOracleReach(`${PROPERTY} (fixed)`, reachStates) - await fc.assert( - fc.asyncProperty( - history, - ([ - top, - nested, - grouping, - shadow, - steps, - union, - bareKeys, - stripped, - alone, - ]) => - runHistory( - top, - nested, - grouping, - shadow, - steps, - union, - bareKeys, - stripped, - alone, - ), - ), - { numRuns: oracleRuns(160), seed: 2094 }, - ) - reach.assertReached() - reach = undefined - }) + oracleReplayTest(it)( + `match the model across generated shapes (fixed campaign)`, + async () => { + // Keep the literal budget/seed binding visible to the run-budget audit. + const options = { numRuns: oracleRuns(160), seed: 2094 } + await runCampaign( + `fixed`, + oracleRandomParameters( + options.numRuns, + options.seed, + undefined, + examples, + ), + ) + }, + ) - it(`match the model across generated shapes (random or replayed)`, async () => { - reach = createOracleReach(`${PROPERTY} (random)`, reachStates) - await fc.assert( - fc.asyncProperty( - history, - ([ - top, - nested, - grouping, - shadow, - steps, - union, - bareKeys, - stripped, - alone, - ]) => - runHistory( - top, - nested, - grouping, - shadow, - steps, - union, - bareKeys, - stripped, - alone, - ), - ), - oraclePropertyOptions(160, PROPERTY), - ) - reach.assertReached() - reach = undefined - }) + oracleReplayTest(it, PROPERTY)( + `match the model across generated shapes (random or replayed)`, + async () => { + await runCampaign( + `random`, + oraclePropertyOptions(160, PROPERTY, { examples }), + ) + }, + ) // Pinned replay of a review counterexample: a grouped select value that // holds the group key inside a function must read the group's key. - it(`reads the group key inside a function`, async () => { - await runHistory( - { fields: { f0: { k: `add`, a: { k: `lit`, n: 0 }, b: { k: `key` } } } }, - { fields: { f0: { k: `agg`, fn: `count` } } }, - `key`, - false, - [], - ) + oracleReplayTest(it)(`reads the group key inside a function`, async () => { + await runHistory({ + top: { + fields: { f0: { k: `add`, a: { k: `lit`, n: 0 }, b: { k: `key` } } }, + }, + nested: { fields: { f0: { k: `agg`, fn: `count` } } }, + grouping: `key`, + }) }) // Pinned replay: a published row keeps the select's field order, also // when a group key comes before a plain aggregate. - it(`keeps the select's field order`, async () => { - await runHistory( - { + oracleReplayTest(it)(`keeps the select's field order`, async () => { + await runHistory({ + top: { fields: { f0: { k: `key` }, f1: { k: `agg`, fn: `count` }, }, }, - { fields: { f0: { k: `agg`, fn: `count` } } }, - `key`, - false, - [], - ) + nested: { fields: { f0: { k: `agg`, fn: `count` } } }, + grouping: `key`, + }) }) // Pinned replay of a generated counterexample: an aggregate query without // groupBy whose select wraps its only aggregate still accepts HAVING. - it(`accepts HAVING beside a wrapped aggregate without groupBy`, async () => { - await runHistory( - { - fields: { - f0: { k: `add`, a: { k: `agg`, fn: `count` }, b: { k: `lit`, n: 1 } }, + oracleReplayTest(it)( + `accepts HAVING beside a wrapped aggregate without groupBy`, + async () => { + await runHistory({ + top: { + fields: { + f0: { + k: `add`, + a: { k: `agg`, fn: `count` }, + b: { k: `lit`, n: 1 }, + }, + }, + having: { part: { k: `lit`, n: 0 }, lit: 0 }, }, - having: { part: { k: `lit`, n: 0 }, lit: 0 }, - }, - { - fields: { - f0: { k: `add`, a: { k: `agg`, fn: `count` }, b: { k: `parent` } }, + nested: { + fields: { + f0: { k: `add`, a: { k: `agg`, fn: `count` }, b: { k: `parent` } }, + }, + having: { part: { k: `parent` }, lit: 1 }, }, - having: { part: { k: `parent` }, lit: 1 }, - }, - `none`, - false, - [], - ) - }) + grouping: `none`, + }) + }, + ) // Pinned replay: a parent field stays legal when the include's source // shadows the parent alias. - it(`reads a parent field under a shadowing child alias`, async () => { - await runHistory( - { fields: { f0: { k: `agg`, fn: `count` } } }, - { - fields: { - f0: { k: `parent` }, - f1: { k: `agg`, fn: `count` }, + oracleReplayTest(it)( + `reads a parent field under a shadowing child alias`, + async () => { + await runHistory({ + top: { fields: { f0: { k: `agg`, fn: `count` } } }, + nested: { + fields: { + f0: { k: `parent` }, + f1: { k: `agg`, fn: `count` }, + }, }, - }, - `none`, - true, - [], - ) - }) + grouping: `none`, + shadow: true, + }) + }, + ) }) diff --git a/packages/db/tests/query/includes-publication-oracle.test.ts b/packages/db/tests/query/includes-publication-oracle.test.ts index 8326dcdfb6..a4f8ad5b6b 100644 --- a/packages/db/tests/query/includes-publication-oracle.test.ts +++ b/packages/db/tests/query/includes-publication-oracle.test.ts @@ -597,7 +597,7 @@ function publicationCampaigns(baseRuns: number, property: string) { if (replay.replayPath !== undefined && replay.replayProperty !== property) return [] const options = oraclePropertyOptions(baseRuns, property) - if (options.seed !== undefined) return [{ name: `replay`, options }] + if (replay.replaySeed !== undefined) return [{ name: `replay`, options }] return [ { name: `fixed seed ${fixedPublicationSeed}`, diff --git a/packages/db/tests/utils.test.ts b/packages/db/tests/utils.test.ts index 24dfc7c3a2..8570b58130 100644 --- a/packages/db/tests/utils.test.ts +++ b/packages/db/tests/utils.test.ts @@ -149,7 +149,7 @@ describe(`oracle run configuration`, () => { expect( oracleRandomParameters(40, ordinaryRun, `includes.incremental-history`), - ).toEqual({ numRuns: 40 }) + ).toEqual({ numRuns: 40, seed: expect.any(Number) }) expect( oracleRandomParameters(40, replayRun, `includes.alpha-renaming`), ).toEqual({ diff --git a/packages/db/tsconfig.aggregate.json b/packages/db/tsconfig.aggregate.json new file mode 100644 index 0000000000..90cba389e5 --- /dev/null +++ b/packages/db/tsconfig.aggregate.json @@ -0,0 +1,22 @@ +{ + "extends": "./tsconfig.test.json", + "compilerOptions": { + "incremental": false, + "noEmit": true, + "paths": { + "@tanstack/db": ["./src/index.ts"], + "@tanstack/db-ivm": ["../db-ivm/src/index.ts"], + "@tanstack/store": ["../store/src/index.ts"] + } + }, + "include": [ + "tests/query/aggregate-select-oracle.property.test.ts", + "tests/declared-reach-oracle.property.test.ts", + "tests/oracle-campaign.test.ts", + "tests/oracle-reach.test.ts", + "tests/oracle-replay.test.ts", + "tests/query/includes-publication-oracle.test.ts", + "../db-ivm/tests/structural-identity-oracle.property.test.ts", + "../offline-transactions/tests/transaction-serializer-oracle.property.test.ts" + ] +} diff --git a/packages/db/vitest.aggregate.config.mjs b/packages/db/vitest.aggregate.config.mjs new file mode 100644 index 0000000000..4ae10b1692 --- /dev/null +++ b/packages/db/vitest.aggregate.config.mjs @@ -0,0 +1,29 @@ +import { fileURLToPath } from 'node:url' + +const path = (relative) => fileURLToPath(new URL(relative, import.meta.url)) + +// A focused source run, with the same jsdom setup as the package suite. +export default { + root: path(`./`), + resolve: { + alias: { + '@tanstack/db-ivm': path(`../db-ivm/src/index.ts`), + '@tanstack/store': path(`../store/src/index.ts`), + }, + }, + test: { + environment: `jsdom`, + setupFiles: [path(`./tests/test-setup.ts`)], + include: [ + `tests/query/aggregate-select-oracle.property.test.ts`, + `tests/declared-reach-oracle.property.test.ts`, + `tests/oracle-campaign.test.ts`, + `tests/oracle-reach.test.ts`, + `tests/oracle-replay.test.ts`, + ], + maxWorkers: 1, + minWorkers: 1, + coverage: { enabled: false }, + typecheck: { enabled: false }, + }, +} diff --git a/packages/offline-transactions/tests/transaction-serializer-oracle.property.test.ts b/packages/offline-transactions/tests/transaction-serializer-oracle.property.test.ts index a4cdcb8bea..eec9fd11d7 100644 --- a/packages/offline-transactions/tests/transaction-serializer-oracle.property.test.ts +++ b/packages/offline-transactions/tests/transaction-serializer-oracle.property.test.ts @@ -5,7 +5,11 @@ import { expect, it, vi } from 'vitest' import { OutboxManager } from '../src/outbox/OutboxManager' import { TransactionSerializer } from '../src/outbox/TransactionSerializer' import { runOracleCampaign } from '../../db/tests/oracle-campaign' -import { oracleReplayTest, oracleRuns } from '../../db/tests/oracle-config' +import { + oracleReplayTest, + oracleRuns, + readOracleRunConfig, +} from '../../db/tests/oracle-config' import { declareOracle } from '../../db/tests/oracle-declaration' import { cleanupOfflineOracle } from './oracle-lifecycle' import { FakeStorageAdapter } from './harness' @@ -561,14 +565,15 @@ oracleReplayTest(it, MUTATION_WIRE_PROPERTY)( // TANSTACK_DB_ORACLE_PROPERTY=offline.serializer.mutation-wire const property = mutationMeaningProperty(`omit-changes`) const replay = mutationWireOracle.random + const { replaySeed } = readOracleRunConfig() const captured = await fc.check(property, { ...replay, - seed: replay.seed ?? 20260914, + seed: replaySeed ?? 20260914, }) expect(captured.failed).toBe(true) expect(captured.errorInstance).toMatchObject({ name: `AssertionError` }) expect(captured.error).toContain(`mutation wire checkpoint`) - if (replay.seed !== undefined) return + if (replaySeed !== undefined) return expect(captured.numShrinks).toBeGreaterThan(0) expect(captured.counterexamplePath).toMatch(/^\d+(?::\d+)*$/) diff --git a/packages/react-native-db-sqlite-persistence/tests/op-sqlite-driver.test.ts b/packages/react-native-db-sqlite-persistence/tests/op-sqlite-driver.test.ts index 36d5fdcdb9..0eda3194a7 100644 --- a/packages/react-native-db-sqlite-persistence/tests/op-sqlite-driver.test.ts +++ b/packages/react-native-db-sqlite-persistence/tests/op-sqlite-driver.test.ts @@ -320,12 +320,12 @@ it.each([ }, ) -it(`configures identical fixed and seedless-random alias campaigns`, () => { +it(`selects a fresh seed without changing the random alias campaign budget`, () => { const runs = readAliasOracleRuns({}) expect(runs).toBe(oracleRuns(50)) expect( oracleRandomParameters(runs, undefined, `op-sqlite.write-marker-aliases`), - ).toEqual({ numRuns: runs }) + ).toEqual({ numRuns: runs, seed: expect.any(Number) }) }) it(`reseeds the random alias campaigns from a seed-only replay`, () => { diff --git a/scripts/oracle-diagnostics.vitest.config.mjs b/scripts/oracle-diagnostics.vitest.config.mjs new file mode 100644 index 0000000000..894d70ed2a --- /dev/null +++ b/scripts/oracle-diagnostics.vitest.config.mjs @@ -0,0 +1,27 @@ +import { fileURLToPath } from 'node:url' + +const path = (relative) => fileURLToPath(new URL(relative, import.meta.url)) + +// These owners distinguish a replay selection from a freshly chosen seed. +export default { + root: path(`../`), + resolve: { + alias: { + '@tanstack/db': path(`../packages/db/src/index.ts`), + '@tanstack/db-ivm': path(`../packages/db-ivm/src/index.ts`), + '@tanstack/store': path(`../packages/store/src/index.ts`), + }, + }, + test: { + environment: `jsdom`, + include: [ + `packages/db-ivm/tests/structural-identity-oracle.property.test.ts`, + `packages/db/tests/query/includes-publication-oracle.test.ts`, + `packages/offline-transactions/tests/transaction-serializer-oracle.property.test.ts`, + ], + maxWorkers: 1, + minWorkers: 1, + coverage: { enabled: false }, + typecheck: { enabled: false }, + }, +}