Skip to content

Index function-call time windows while preserving pandas query semantics - #147

Open
Essoz wants to merge 7 commits into
codex/inference-identity-cachefrom
codex/inference-time-range-index
Open

Essoz wants to merge 7 commits into
codex/inference-identity-cachefrom
codex/inference-time-range-index

Conversation

@Essoz

@Essoz Essoz commented Oct 25, 2026 •

Copy link
Copy Markdown
Collaborator

Repeated function-call time-window queries scan all event rows before grouping, including full-column type membership tests. Add a lazy process/thread timestamp index for conservative native numeric cases, then restore original row order and retain the existing pandas grouping, assertions, and fresh record construction. Unsupported dtypes, mixed-precision/scalar comparisons, and malformed queries fall back to the original mask.

This PR is stacked on #144. The full-trace experiment applies the separate #145 pruning-state correction to both variants; that correctness change is not bundled in this PR.

Measured result

Three sequential alternating pairs on the complete saved tiny-Llama trace (573,494 events, six enabled relations, one CPU worker/native thread):

Pair/order Baseline Indexed
1: baseline, candidate 102.655 s 76.748 s
2: candidate, baseline 101.796 s 76.788 s
3: baseline, candidate 101.610 s 76.658 s

Median 101.796 → 76.748 s: 1.326×, 24.61% less wall time. All six complete outputs match the corrected reference: 836 invariants and 104 failed-hypothesis records, with arrays and multiplicity preserved. Raw output and full/stage context files are also byte-identical across variants. Source/input integrity checks passed; independent audit verified the retained source/output/context evidence and recomputed the statistics.

Median sampled child-session PSS increased 2.129 → 2.168 GiB (+40.15 MiB). The first baseline peak was higher than later baseline peaks; all per-run peaks are retained in the report. Fresh-process timings include lazy index construction. A separate 200k-row/1k-query synthetic benchmark measures 2.662 → 0.340 s, with first-query build cost shown separately.

Validation and limits

  • 18 new differential tests; 61 combined local tests and 67 in the matching Linux runtime, including six shared Isolate pruning state between independent precondition solves #145 tests.
  • Independent parent-method oracle: 144 dtype/offset cases and 6,480 queries in each of two pandas/NumPy runtimes, zero differences.
  • Pinned Ruff, Mypy, Black, and isort checks pass. Production and benchmark checkpoints are signed.
  • Conservative fallbacks cover dtype/null/category behavior, numeric precision, strict/reversed bounds, incomplete/duplicate/exception records, and unusual scalar comparisons.
  • Cache contract requires stable time/type/PID/TID values, dtypes, and row order. Frame replacement or length changes invalidate it; same-size edits require a new TracePandas. Storage grows O(N).
  • Shared host, three pairs on one saved trace, six enabled relations, no CPU affinity; this does not establish training overhead or distributed scaling. CPU Torch matches the original CUDA wheel's source revision, but Python/build differ. Do not multiply this incremental result by the earlier single-pair cache result.

Measurement report and compact provenance include reproduction instructions, all run timings, memory tradeoffs, source/trace/output digests, and retained archive identity. Raw trace/output records remain private study artifacts.

Engineering hardening

Typed numeric time-index storage and query boundaries, retaining conservative dtype/scalar validation and exact fallback semantics (8dc1fd1), plus signed parent hardening updates. All 97 affected inference tests, all three benchmark smoke checks, pinned hooks, and six stronger Mypy 1.9 module/tool/test scopes pass. A new bounded replay of source ce25c12 plus the common #145 production patch emits all 836 invariants and 104 failed hypotheses byte-for-byte identical to the corrected reference; both context files are also identical. The verification run took 76.794 seconds, without a new paired speedup claim. Exact source/runtime/equality evidence is in docs/measurements/inference-hardening-validation-20261025.json.

Separately composed updated production sources pass 140 affected Linux regression tests, whole-source Mypy (77 files), Ruff, and isort. Whole-source Black still reports three unchanged legacy files, down from six in the original base; changed-file hooks pass. Strict checks are scoped and do not establish repository-wide strict typing. Historical performance numbers above retain their original measured revisions; this hardening pass does not re-estimate them.

@Essoz
Essoz marked this pull request as ready for review October 25, 2026 11:03

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant