Skip to content

Cache process and thread identities during precondition inference - #144

Open
Essoz wants to merge 6 commits into
codex/inference-indexesfrom
codex/inference-identity-cache
Open

Essoz wants to merge 6 commits into
codex/inference-indexesfrom
codex/inference-identity-cache

Conversation

@Essoz

@Essoz Essoz commented Oct 25, 2026 •

Copy link
Copy Markdown
Collaborator

Repeated get_meta_vars calls rescanned both full process/thread columns for each precondition example. Cache the exact pandas dropna().unique().tolist() results lazily per trace, return fresh lists, and rebuild when the frame is replaced or its length changes.

On the complete saved 573,494-event tiny reference trace, total wall time fell from 1,385.656 s to 98.551 s (14.06x). This is one sequential baseline-first CPU pair on labmach, with one worker/native thread and the same six enabled relations. Both runs completed; all 835 invariant records and 105 saved failed-hypothesis records match as complete structural JSON multisets, preserving duplicate occurrences and array order. All source/input hashes stayed unchanged, and both full/stage context files match byte-for-byte. Peak sampled child-session PSS stayed essentially unchanged: 2.127 → 2.130 GiB.

Phase Baseline Cache
Trace loading 10.503 s 10.699 s
Reported inference 1,371.032 s 83.717 s
Generation/pruning within inference 619.580 s 72.759 s
Final preconditions/sort within inference 751.403 s 10.913 s

Stacked on #141 (codex/inference-indexes). The isolated synthetic benchmark also improves construction plus 1,000 metadata queries from 9.489831 s to 0.012004 s median across three alternating repetitions, with all ordered results matching. It is distinct from the full inference measurement above.

Validation: 43 local context/call/identity tests, 11 identity tests in the Linux replay runtime, and repository-pinned Ruff/Mypy/Black/isort hooks pass. Independent checks cover 78 comparisons across 13 dtype/value cases, six pickle protocols, actual spawned workers, and all archived full-trace evidence. Reproduce the portable microbenchmark with python -m tools.benchmarks.identity_cache; measurement details, phase definitions, hashes, and runtime provenance are in docs/measurements/inference-identity-cache.md and its linked JSON evidence.

Limits: identity values, dtypes, and row order must remain stable within a frame; same-size in-place edits, column replacement, or reordering require a new TracePandas. The full-trace result is one unprofiled pair, not a repeated-run estimate, and excludes FunctionLeadRelation/FunctionCoverRelation. CPU replay uses the original Torch source revision but a different Python/device build. It does not establish training overhead, distributed scaling, memory savings, or invariant ground truth. The separate pruning-state correctness fix #145 is excluded from both performance variants. Two earlier cProfile-plus-periodic-traceback attempts crashed and are excluded from all performance claims; the successful pair alone does not establish the crash cause.

Engineering hardening

Typed identity caches and benchmark results (c125d06) and incorporated signed parent hardening updates. Pandas identity getters now truthfully return list[object]; the abstract backend contract is Sequence[object], preserving integer backend overrides. Caller review found membership use rather than numeric assumptions. Scalar values, order, duplicate behavior, fresh-list ownership, and the documented stable-frame cache contract are unchanged. All 43 context/call/identity tests, the benchmark smoke check, pinned hooks, and stronger Mypy 1.9 checking of the benchmark pass.

Separately composed updated production sources pass 140 affected Linux regression tests, whole-source Mypy (77 files), Ruff, and isort. Whole-source Black still reports three unchanged legacy files, down from six in the original base; changed-file hooks pass. Strict checks are scoped and do not establish repository-wide strict typing. Historical performance numbers above retain their original measured revisions; this hardening pass does not re-estimate them.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant