Skip to content

Reduce repeated instrumenter metadata lookup and scalar JSON probes - #163

Merged
Essoz merged 3 commits into
codex/metadata-method-fastpathfrom
codex/instrumenter-hotpath-20261028
Oct 30, 2026
Merged

Essoz merged 3 commits into
codex/metadata-method-fastpathfrom
codex/instrumenter-hotpath-20261028

Conversation

@Essoz

@Essoz Essoz commented Oct 27, 2026 •

Copy link
Copy Markdown
Collaborator

Every Tensor metadata conversion repeats stable base-class lookup checks, and
scalar arguments undergo a throwaway JSON serialization before their final
serialization. This change caches only immutable lookup-hook facts and skips the
compatibility probe for exact None/bool/float/str and signed-64-bit integers.

All descriptors and mutable class/instance state remain live. The cache adds no
references to application descriptors or their names; a regression checks the
observable reference counts. Existing field selection, getter order, warnings,
method-binding lifetimes, compile/FakeTensor handling, larger-integer failures,
snapshot timing, JSON encoding and trace records are preserved.

Ordinary labmach CPU comparisons use four fresh runs per revision in ABBA×2
order, one pinned performance core and one native thread. Full API tracing is
enabled; model tracking is disabled for timing. These are measured operation/
training-body reductions, not GPU or total process-runtime claims.

Workload Baseline Candidate Less time
800 add/mm calls, Tensor + Parameter 1.516760 s 1.479685 s 2.44%
MNIST, 10 steps 0.667927 s 0.646459 s 3.21%
Transformer, 10 steps 1.771281 s 1.725027 s 2.61%

One paired operation trial and one paired MNIST trial were slower; the report
preserves every sample.
The scalar component alone improves 35–37% on two local series, but that is not
the training speedup. Fixed cache containers are 344 B/process in the local
runtime, excluding code/class objects and allocator overhead. Python-validated
pickle capture, guarded JSON fallback bypass, and byte batching did not justify
a production change; their measurements and limits are in the report.

Validation: 70 focused Python 3.10 regressions and 55 Python 3.12 tests pass;
configured changed-file hooks, explicit new-test lint/format and strict scoped
mypy pass. Full suite: 357 pass with two failures independently reproduced on the
unchanged baseline (test_traincheck_internal_attr_hidden, test_inject_eval_loop).
All 24 instrumented operation/training runs match the recorded plain state,
losses and RNG. Twelve complete ordered raw comparisons retain every ID/time
field and find only those process-specific differences. Separate three-step
combined API+subclass-model-tracker runs also match recorded state/loss/RNG and
API/VAR payloads. Existing Transformer pre/post observer coverage remains partial
and is documented; no invariant checker ran. H100 validation is deferred until
access returns; no GPU or online-checker improvement is claimed.

See the measurement report
for raw evidence locations, commands, source/harness hashes, memory/drain scope,
repeat timings, output comparison limits and rejected experiments.

Dependency: targets codex/metadata-method-fastpath (#162).
Baseline: bccc56fab83d525198286f1855b9dd7f6866135f;
corrected implementation: 161313a93512b07410fa2b1072613333a5a2e492.
Earlier f9b221f timings are historical and are not used for the final results.
Commits are signed. No merge has been performed.

@Essoz
Essoz marked this pull request as ready for review October 27, 2026 18:10
@Essoz
Essoz merged commit 847a0bd into codex/metadata-method-fastpath Oct 30, 2026
0 of 2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant