Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
72 commits
Select commit Hold shift + click to select a range
34ac80e
Make inference sampling independent of worker scheduling
Essoz Oct 13, 2026
6c1a5d4
Fix bounded reservoir construction and population-aware merges
Essoz Oct 25, 2026
f6390cd
Combine bounded reservoirs with deterministic inference sampling
Essoz Oct 25, 2026
1eb17b7
Group context-manager events for invariant inference initialization
Essoz Oct 25, 2026
d144015
Index pre and post call records by ID during inference
Essoz Oct 25, 2026
7d6c683
Fix recorder re-entry and atomic trace buffer delivery
Essoz Oct 25, 2026
504e04d
Cache process and thread identities during precondition inference
Essoz Oct 25, 2026
9ac61d1
Preserve TorchScript builtin identity through tracing wrappers
Essoz Oct 25, 2026
fa27743
Document identity cache measurements and validation
Essoz Oct 25, 2026
1b5aedf
Isolate pruning state between independent precondition solves
Essoz Oct 25, 2026
99d1690
Document complete identity-cache inference comparison
Essoz Oct 25, 2026
64af67b
Document pruning-state replay output changes
Essoz Oct 25, 2026
24e7cb0
Index function-call time windows without changing pandas grouping
Essoz Oct 25, 2026
39152ad
Add alternating time-window query benchmark with build costs
Essoz Oct 25, 2026
2f034ae
fix: validate dynamic type-name metadata before fallback
Essoz Oct 25, 2026
74b3adf
Add fresh-process invariant validation harness
Essoz Oct 25, 2026
e673266
Document dynamic type-name validation on matching Linux runtime
Essoz Oct 25, 2026
927707d
Document audited time-window inference measurements and tradeoffs
Essoz Oct 25, 2026
cad62cc
Validate a live dtype rule when no shape control is available
Essoz Oct 25, 2026
2333682
Record successful collection with the narrow type-name fix
Essoz Oct 25, 2026
27e67aa
Document fresh-run validation and object-identity false alarms
Essoz Oct 25, 2026
aec748e
test: type pruning-state regression contracts
Essoz Oct 25, 2026
c638566
Type deterministic sampling and inference task contracts
Essoz Oct 25, 2026
3866d1a
Merge branch 'codex/deterministic-inference' into codex/inference-res…
Essoz Oct 25, 2026
ba52686
Define reservoir worker configuration and mutation contracts
Essoz Oct 25, 2026
9c9bc17
Merge branch 'codex/inference-reservoir-cap' into codex/inference-ind…
Essoz Oct 25, 2026
aba4ee8
Type tracing suppression and buffer failure contracts
Essoz Oct 25, 2026
0cb2f14
Merge branch 'codex/trace-integrity' into codex/torchscript-builtin-i…
Essoz Oct 25, 2026
336fc1b
Specify context index and benchmark data contracts
Essoz Oct 25, 2026
1ae1d12
Merge branch 'codex/inference-indexes' into codex/inference-identity-…
Essoz Oct 25, 2026
c125d06
Describe schemaless identity values and benchmark results honestly
Essoz Oct 25, 2026
6f53ba2
Merge branch 'codex/inference-identity-cache' into codex/inference-ti…
Essoz Oct 25, 2026
fa4e498
Validate complete worker snapshots before changing process state
Essoz Oct 25, 2026
5295ffd
Merge branch 'codex/inference-reservoir-cap' into codex/inference-ind…
Essoz Oct 25, 2026
ec4fbe5
Merge branch 'codex/inference-indexes' into codex/inference-identity-…
Essoz Oct 25, 2026
d4ebb12
Enforce TorchScript compatibility metadata contracts
Essoz Oct 25, 2026
9bfcb17
Merge branch 'codex/torchscript-builtin-identity' into codex/dynamic-…
Essoz Oct 25, 2026
8dc1fd1
Type validated numeric time indexes and benchmark measurements
Essoz Oct 25, 2026
ce25c12
Merge branch 'codex/inference-identity-cache' into codex/inference-ti…
Essoz Oct 25, 2026
1a05921
Document tracing contract hardening validation
Essoz Oct 25, 2026
94850f7
Merge branch 'codex/trace-integrity' into codex/torchscript-builtin-i…
Essoz Oct 25, 2026
7bba403
Document TorchScript contract boundaries and validation
Essoz Oct 25, 2026
b91363f
Merge branch 'codex/torchscript-builtin-identity' into codex/dynamic-…
Essoz Oct 25, 2026
b64023b
Type dynamic name contracts and retain failure assertions
Essoz Oct 25, 2026
409d57c
docs: record hardened inference replay parity
Essoz Oct 25, 2026
f3804c0
Harden fresh-validation typing and report contracts
Essoz Oct 25, 2026
db80c19
Record observer contract regression evidence
Essoz Oct 25, 2026
91d151e
docs: verify hardened observer against updated PR stack
Essoz Oct 25, 2026
6759bd6
Merge reconciled parallel-infer-integration base into PR #140
Essoz Oct 26, 2026
0866977
Merge reconciled inference-reservoir-cap base into PR #141
Essoz Oct 26, 2026
094182d
Merge reconciled parallel-infer-integration base into PR #142
Essoz Oct 26, 2026
f37be1e
Merge reconciled trace-integrity base into PR #143
Essoz Oct 26, 2026
d705073
Merge reconciled parallel-infer-integration base into PR #145
Essoz Oct 26, 2026
47dc072
Merge reconciled inference-indexes base into PR #144
Essoz Oct 26, 2026
43fe506
Merge reconciled torchscript-builtin-identity base into PR #146
Essoz Oct 26, 2026
9b43ff0
Merge reconciled inference-identity-cache base into PR #147
Essoz Oct 26, 2026
62b829e
Merge reconciled inference-identity-cache base into PR #148
Essoz Oct 26, 2026
dc4bf7d
Merge pull request #140 from OrderLab/codex/inference-reservoir-cap
Essoz Oct 26, 2026
dd33de9
Merge pull request #141 from OrderLab/codex/inference-indexes
Essoz Oct 26, 2026
cf93345
Merge pull request #142 from OrderLab/codex/trace-integrity
Essoz Oct 26, 2026
959c93a
Merge pull request #143 from OrderLab/codex/torchscript-builtin-identity
Essoz Oct 26, 2026
6d1d7c9
Merge pull request #144 from OrderLab/codex/inference-identity-cache
Essoz Oct 26, 2026
722e16c
Merge pull request #145 from OrderLab/codex/precondition-state-isolation
Essoz Oct 26, 2026
dc677be
Merge pull request #146 from OrderLab/codex/dynamic-type-names
Essoz Oct 26, 2026
4d60091
Merge pull request #147 from OrderLab/codex/inference-time-range-index
Essoz Oct 26, 2026
04d3325
Merge pull request #148 from OrderLab/codex/fresh-run-invariant-valid…
Essoz Oct 26, 2026
527164c
Integrate reviewed parallel-infer-integration changes for main
Essoz Oct 26, 2026
4fa2e55
Integrate reviewed inference-reservoir-cap changes for main
Essoz Oct 26, 2026
1454ffa
Integrate reviewed inference-indexes changes for main
Essoz Oct 26, 2026
95ff5bf
Integrate reviewed trace-integrity changes for main
Essoz Oct 26, 2026
338f21d
Integrate reviewed torchscript-builtin-identity changes for main
Essoz Oct 26, 2026
caabdd7
Integrate reviewed inference-identity-cache changes for main
Essoz Oct 26, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
67 changes: 67 additions & 0 deletions docs/experiments/dynamic-type-names-results.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
{
"baseline_commit": "9ac61d1140cf8679c24442a174ec12f4de069980",
"python": "3.10.16",
"torch": "2.2.2",
"platform": "macOS-27.0.1-arm64-arm-64bit",
"method": "Identical seven-test suite, candidate module globals, exact parent typename function extracted by AST versus candidate function.",
"before": {
"tests": 7,
"failures": 3,
"errors": 0,
"passed": 4,
"failed_tests": [
"test_dynamic_typename.DynamicTypenameTests.test_dynamic_callable_metadata_uses_class_name",
"test_dynamic_typename.DynamicTypenameTests.test_non_string_name_values_fall_back_without_truth_testing",
"test_dynamic_typename.DynamicTypenameTests.test_real_pytorch_dynamic_ops_wrapper"
]
},
"after": {
"tests": 7,
"failures": 0,
"errors": 0,
"passed": 7,
"failed_tests": []
},
"linux_followup": {
"fix_commit": "2f034ae5d0fab7e792b7c0a5bdc6a1acfeb8962c",
"utils_sha256": "854c11519e7aa6d0cbbfd79694d2ad4895fb1aaf72e9efbb0068048598b70750",
"runtime": {
"python": "3.11.4 (main, Jul 5 2023, 13:45:01) [GCC 11.2.0]",
"torch": "2.16.0.dev20261012+cpu"
},
"ops_wrapper_names": {
"instrumentation": "torch._inductor.virtualized.OpsWrapper",
"runtime": "torch._inductor.virtualized.OpsWrapper"
},
"tests": {
"v3-integration-tests": {
"tests_passed": 29,
"exit_code": 0,
"source_unchanged": true,
"stdout_sha256": "be7f9c929651de7f09f383a8f3b0176ec8d3285f0ad053f4d49e6e837a40309b"
},
"v3-coverage-tests": {
"tests_passed": 11,
"exit_code": 0,
"source_unchanged": true,
"stdout_sha256": "a33575d7feb6117b55c84c6919d21911e1f76065db033ebbdd9270671c536239"
}
},
"scope": "Exact PR146 utility in composed source; seven type-name tests included in 29 integration tests. No training or timing claim from these preflight results.",
"collection_smoke": {
"workload": "Bounded benchmark MNIST Net / cached real MNIST / batch2 / CPUfloat32 / seed42",
"completed_steps": 10,
"success": true,
"source_unchanged": true,
"state_shapes_unchanged": true,
"source_stack": "PR144 plus PR142/143/145/146 production fixes; immutable integration-v3, signed74b harness",
"evidence_sha256": {
"phase-result.json": "03a50887287fb8e98398735e738412edd6c356bcc21dc39ee15929d5ffcf324c",
"metrics.json": "d99a6fd45dc534964835937a8a623ad98309052a3771e9750c3f8dfb0bfe0713",
"adaptation.json": "8a533afe8ea099fbb255124ba589f2edf068f1c17d5d3774d73c0ff88a63fa8e",
"invocation.json": "8ecea3da01502bff67024e8aa303f3d7e6ab72e7a7d1191b3aef53d61868fc75"
},
"scope": "Collection completed with exact narrow PR146 utility in composed stack. Separate later shape-only fault-selector failure does not affect this collection result. No fix-isolated training timing claim."
}
}
}
101 changes: 101 additions & 0 deletions docs/experiments/dynamic-type-names.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,101 @@
# Dynamic type-name metadata

## Problem and change

PyTorch `torch._inductor.virtualized.ops` returns callable values for dynamic
`__qualname__` and `__name__` attributes. TrainCheck rejected the first value but
accepted the second, then asserted that its type name was a string. A fresh
MNIST trace-collection attempt failed before training reached its first step.

Validate `__name__` as a nonempty string before using it. Otherwise, use the
existing reported-class fallback. This does not call dynamic metadata, truth-test
non-string values, change valid name precedence, or replace the reported
`__class__` with the object's actual type.

## Validation

An identical seven-test suite was run against the exact parent `typename`
function extracted from commit `9ac61d1140cf8679c24442a174ec12f4de069980` and against
the changed function, with identical module globals. Python 3.10.16 / PyTorch
2.2.2 on macOS arm64:

| Result | Parent | Changed |
| --- | ---: | ---: |
| Tests passed | 4 | 7 |
| Tests failed | 3 | 0 |
| Test errors | 0 | 0 |

The three previously failing tests cover callable dynamic metadata, non-string
names (including an unsafe truth-value implementation), and the real PyTorch
`OpsWrapper`. The remaining tests preserve qualified/simple names, reported-class
proxies, functions, descriptors, tensors, and parameters. Both instrumentation
and runtime modes are exercised for dynamic metadata.

Reproduce the candidate suite with:

```sh
python -m unittest tests.test_dynamic_typename -v
```

The first successful Linux training pilot used an existing broader temporary
workaround. That pilot is not evidence for this narrower production change; the
follow-up below uses the exact signed fix. This is a compatibility fix, with no
performance improvement claim.

## Matching Linux runtime follow-up

The exact signed utility change was also tested on Python 3.11.4 and PyTorch
2.16.0.dev20261012+cpu. All seven type-name regressions passed within a **29-test
integration suite** (also covering trace integrity, TorchScript, and precondition
state isolation). A separate 11-test checker-observer suite passed. The real
`OpsWrapper` returned `torch._inductor.virtualized.OpsWrapper` in both modes.

The disposable integration source combined the signed tracing, TorchScript,
cache, sampling, state-isolation, and type-name changes. Its `utils.py` matched
commit `2f034ae5d0fab7e792b7c0a5bdc6a1acfeb8962c` exactly, and all source hashes
were unchanged after testing. Compact evidence includes runtime versions and
log/source hashes. These preflight results do not establish training overhead
or cross-workload invariant generalization.

The composed source also completed a fresh **10-step MNIST trace collection**
with cached real MNIST, batch size 2, CPU float32, and seed 42. Its source hashes
and model-state shapes stayed unchanged. This supports collection compatibility
in the identified stack; it does not isolate training behavior to this one fix.
The study later stopped at a shape-only negative-control selector because no
shape invariant existed, after collection and healthy checks had succeeded.
That harness issue and its preserved evidence are separate from the type-name
fix. Evidence hashes for the successful collection are included in the JSON.

## Contract hardening (2026-10-25)

`typename` now declares its existing `object`/`bool` inputs and string result.
The dynamic-name guards, reported-class fallback, and final string assertions
are unchanged. All dynamic-name test helpers now have explicit annotations.
Two additional tests preserve the negative contract: an invalid fallback class
name still fails the final assertion, and an unexpected metadata exception
propagates unchanged. This does not change `safe_getattr`'s pre-existing framework
compatibility exception handling.

The composed branch includes the signed #142 and #143 hardening checkpoints.
Validation uses Python 3.10.16/PyTorch 2.2.2 with one OpenMP/MKL/OpenBLAS thread:

```sh
python -m unittest discover -s tests -p 'test_dynamic_typename.py'
python -m unittest discover -s tests -p 'test_torchscript*.py'
python -m unittest discover -s tests -p 'test_trac*.py'
pre-commit run --files traincheck/utils.py tests/test_dynamic_typename.py traincheck/instrumentor/tracer.py traincheck/instrumentor/dumper.py traincheck/instrumentor/torchscript_compat.py tests/test_torchscript_compat.py tests/test_torchscript_contracts.py tests/test_tracing_contracts.py
mypy --follow-imports=silent --ignore-missing-imports --check-untyped-defs --disallow-untyped-defs --disallow-any-generics --warn-return-any traincheck/instrumentor/torchscript_compat.py tests/test_dynamic_typename.py tests/test_torchscript_contracts.py tests/test_tracing_contracts.py
```

All 37 tests passed: 9 dynamic-name, 13 TorchScript (including six fresh-process
controls), and 15 tracing tests. All pinned hooks and the stronger scoped check
passed using mypy 1.9. The existing general utility/serialization modules are
not claimed to be strictly typed in full; see the #143 evidence for the remaining
legacy tracer/dumper diagnostics. No runtime benchmark was rerun for this
annotation/contract update, so earlier performance numbers retain their original
revision and workload scope.

With `--check-untyped-defs`, `utils.py` reports the same single pre-existing
`threading.__excepthook__` stub diagnostic at both original head
`2333682cb2f3f48d338ff27a6d70682f0f4e4702` and this update; no new diagnostic
was introduced by the `typename` annotation.
Loading
Loading