Skip to content

Make inference sampling independent of worker scheduling - #139

Merged
Essoz merged 2 commits into
daniel/parallel-inferfrom
codex/deterministic-inference
Oct 25, 2026
Merged

Essoz merged 2 commits into
daniel/parallel-inferfrom
codex/deterministic-inference

Conversation

@Essoz

@Essoz Essoz commented Oct 13, 2026 •

Copy link
Copy Markdown
Collaborator

Process-based inference previously shared or inherited mutable RNG state. Changing worker count or task scheduling could therefore change sampled evidence and support counts, even for the same saved traces. This gives generation, cross-trace collection, and precondition tasks deterministic RNG streams keyed by their semantic identity. Serial and process paths use the same scopes. Precondition downsampling keeps its existing seed-42 sample choices without reseeding the application's global RNG.

Targets daniel/parallel-infer so this PR contains only the sampling fix and its regression tests. Sampling rates and caps are unchanged; moving to task streams can change the selected subset once compared with the previous serial implementation.

Validation:

  • All 27 saved-trace runs agreed on full invariant and failed-hypothesis JSON within each workload: MNIST, Transformer, and DCGAN × 1/2/4 workers × three repetitions. Comparison includes support counts and duplicate multiplicity; only object-key and file-line order are ignored.
  • 11 Linux regression tests pass on the final PR files, including actual containment sampling with 1,005 nested calls per trace, fork and spawn, reversed/repeated dispatch, task/thread isolation, frozen exceptions, stable digest seeds, and exact legacy precondition samples above 5,000 examples.
  • Commit hooks for Ruff, Mypy, Black, and isort pass, as does git diff --check.
  • Old serial versus fixed: predicates and failed hypotheses are unchanged on all three workloads. One positive-support count changes from 20 to 21 in Transformer and one in DCGAN; each is consistent across all patched runs.

The real-workload matrix used Python 3.10.16, PyTorch 2.2.2, pandas 2.3.3, fork, and fixed PYTHONHASHSEED=0. This establishes agreement for the tested inputs and worker counts; it does not claim whole-engine determinism across Python versions, hash seeds, or reordered inputs. Existing set-population ordering and reservoir merge/cap defects remain separate. Two macOS subprocess cases hit OpenMP shared-memory initialization failures; the complete suite passed on Linux.

Reproduce the regression suite:

PYTHONDONTWRITEBYTECODE=1 OMP_NUM_THREADS=1 MKL_NUM_THREADS=1 OPENBLAS_NUM_THREADS=1 \
  python -m unittest discover -s tests -p test_inference_sampling.py -v

Engineering hardening

Typed the semantic sampling identity, RNG protocol, context token/exception lifecycle, task payloads, worker results, and shared helper interfaces in signed commit c638566. All 11 sampling regressions pass; pinned hooks and stronger Mypy 1.9 checks for the new sampling module pass. No RNG stream or sample-selection behavior was changed by this annotation pass.

Separately composed updated production sources pass 140 affected Linux regression tests, whole-source Mypy (77 files), Ruff, and isort. Whole-source Black still reports three unchanged legacy files, down from six in the original base; changed-file hooks pass. Strict checks are scoped and do not establish repository-wide strict typing. Historical performance numbers above retain their original measured revisions; this hardening pass does not re-estimate them.

@Essoz
Essoz merged commit 122e769 into daniel/parallel-infer Oct 25, 2026
0 of 2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant