Skip to content

Bound example reservoirs across construction and inference merges - #140

Merged
Essoz merged 8 commits into
codex/parallel-infer-integrationfrom
codex/inference-reservoir-cap
Oct 26, 2026
Merged

Essoz merged 8 commits into
codex/parallel-infer-integrationfrom
codex/inference-reservoir-cap

Conversation

@Essoz

@Essoz Essoz commented Oct 25, 2026 •

Copy link
Copy Markdown
Collaborator

Example construction and stage/cross-trace merges bypass the configured reservoir cap and leave num_seen stale. With a cap of 2, merging two ten-example streams retains 4 examples and reports only 10 seen; this change retains 2 and reports all 20.

Stream iterable construction through bounded insertion, merge reservoirs using their represented populations, and propagate capacity/seed settings to spawned workers. Separate advancing RNG streams keep nested merges and subsequent additions from replaying selection draws. Complete streams that fit preserve concatenation order and duplicate occurrences.

Validation:

  • 21 reservoir regression tests pass independently; 32 tests pass together with Make inference sampling independent of worker scheduling #139, including actual serial/fork/spawn inference with 1/2/4 workers.
  • Nine complete saved-trace replays (MNIST, Transformer, DCGAN × 1/2/4 workers) match full invariant and failed-hypothesis JSON across workers and against the sampling-only outputs. Input/source hashes are unchanged.
  • The five-item iterable fixture also drops from 5 retained examples to the configured 2 while keeping num_seen=5.

Existing equally seeded source reservoirs can be correlated: the merge preserves marginal inclusion probabilities, without claiming a uniform joint sample. Optional containment generalization is skipped for sampled inputs because discarded set overlap is unknown; this can change generalized hypotheses above the cap. The change bounds each retained reservoir, not total process memory.

Stacked on #150 (codex/parallel-infer-integration), which contains the merged #139 changes. Reproduction details and compact evidence are in docs/experiments/reservoir-cap-validation.md.

Engineering hardening

Typed worker configuration and reservoir mutation interfaces, then added runtime validation of every required worker setting before changing configuration, trace references, or progress state (ba52686, fa4e498). Missing, unknown, and malformed fields fail explicitly; supported stdlib seed types and capacities are preserved. The new contract tests expose 18 failures on the pre-validation implementation; all 36 sampling/reservoir/config tests now pass. Pinned hooks and stronger scoped Mypy 1.9 checks pass.

Separately composed updated production sources pass 140 affected Linux regression tests, whole-source Mypy (77 files), Ruff, and isort. Whole-source Black still reports three unchanged legacy files, down from six in the original base; changed-file hooks pass. Strict checks are scoped and do not establish repository-wide strict typing. Historical performance numbers above retain their original measured revisions; this hardening pass does not re-estimate them.

Integration with current main

Based on #150 (codex/parallel-infer-integration), with signed merge checkpoint 6759bd64decd. The shared base is #150, which reconciles Daniel's parallel-inference tip 122e769 with main ed54c7d. History is preserved; no force push or GitHub PR merge was used. The sampling-test add/add conflict from #139's rewritten merge ancestry retains this PR's uncapped fixture, so the >5,000-example sampling path remains covered. All nine PR heads merge cleanly with their declared bases and current main, and the full stack composes without conflicts.

The resolved composition passes 140 production + 28 harness tests locally (Python 3.10.16 / Torch 2.2.2), pinned Mypy over 77 production files, and stricter Mypy over 9 harness/test files. Conflict-file commit hooks pass. Full-stack Ruff/Mypy/isort pass; Black retains the same three pre-existing formatting failures (config.py, proxy_wrapper/hash.py, proxy_wrapper/subclass.py). Remote CI is ignored because the self-hosted runner is unavailable. These checks validate integration; existing performance measurements retain their original revisions and are not remeasured here.

Land the parent in main first, then retarget this PR to main before merging it. Repeat for its children. Use merge commits to preserve stack ancestry, or refresh descendants after a squash/rebase merge. Exact heads and local logs are retained in outputs/conflict-resolution-20261026/.

@Essoz
Essoz changed the base branch from codex/deterministic-inference to codex/parallel-infer-integration October 26, 2026 08:48
@Essoz
Essoz merged commit dc4bf7d into codex/parallel-infer-integration Oct 26, 2026
0 of 3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant