[midhatn]

Problem and result

Local agents need memory for compilers and GPU tools between model calls. This opt-in draft lets a supervisor declare that work, request resident relief or confirmed inference unload, and return after a fresh capacity check. Sustained external pressure can also trigger relief. Uncertain execution or teardown keeps the ownership barrier in place instead of repeating a tool or regrowing inference over it.

This remains a draft. Current head is 06ba725d8b9bb42b457302f71dd85204e9760c32, incorporating upstream v0.2.3.1 without rewriting the earlier public history. The earlier native executable qualification below applies to a69684f1; b580e627 recorded that result, and 06ba725d adds the Python routing correction described below. Native sources are unchanged. Reopening after the tracker reset did not mark the feature ready for review.

Related discussion

This continues #1493 following the tracker-reset invitation. The reset was not a technical rejection or approval of the scope; no separate maintainer scope agreement is recorded here. The allocator foundation is medking82's #726. The engine-independent Hermes terminal authorization seam is Hermes #135418. The separate Windows token-default correction is #3399.

Approach

Reuse the existing live RAM blocks, GPU VMM cache, acknowledged memory protocol and server lifecycle/FIFO. An authenticated loopback lease separates admission, execution, renewal and release; a supervisor grants tool profiles and runtime-identified workers share one broker. Physical RAM, commit headroom and native GPU headroom are checked separately. The feature is opt-in; unsupported combinations refuse startup rather than bypassing adaptive ownership. Capacity is observed, not reserved by the OS.

The upstream merge retains current vision diagnostics, tool-picture handling, drafting configuration and web API-key behavior while preserving adaptive admission and confirmed process shutdown. Startup validation uses effective user arguments. Protocol and limits and implementation provenance retain behavior and attribution. The private disk checkpoint bank and the newer local Python scheduling experiment are excluded.

Validation

Passed:

• Head 06ba725d: correct delivery of the coadaptive route through the existing native request serializer's strata_tune.pcie_frac field. Explicit nested values, including zero, take precedence and other tuning values remain intact. The new actual-Service/serializer regression fails on preceding public b580e627 (three tests, four failed subtest/assertion results). After the nine-line production correction, all 46 focused routing/coadaptive tests pass. These are CPU/fake-engine checks, not new model or native-build qualification. The independent Windows token-rights fix [Other] preserve Windows child identity after adopting user defaults #3399 and the larger scheduling/learner experiment remain excluded.

• Exact source a69684f1f2609da46c82f9719d5159f5f9b307dc: Windows CUDA 13.4, MSVC 14.44.35207, architecture 89, portable Release build. All 183 build steps completed with one worker in 961.907s. All 13 native utility, protocol and parity programs exited zero, including live_memory_test --protocol-only. No model was started. Configure, build, tests and outer worker exited zero; all 1,638 retained process handles signaled exit and owned cleanup was verified. The strata.exe SHA-256 is 744205e4e024b5cbba19e4b560678ffe21e6899e59050929f7af51557f3c6198.

• Resolved upstream merge e1242813 (upstream 7ddc17a3, source tree b37e52d2): 14 focused server/web modules, 188 collected, 187 passed, one optional jsonschema skip, in 39.078s including wrapper overhead. A separate terminal drafting run passed 40 tests in 7.746s. The current report lists commands, module counts, exact identities, cleanup scope and the compact native receipt.

• Earlier v0.2.2 source 037314f9 plus the five-line test-only synchronization committed as e39dbb99: python -B -m unittest serve.test_idle_parking_service -v, 42 passed in 4.920s. The client could receive its response before the server's finally released preparation ownership; the bounded wait preserves the original assertions. This focused pass does not replace the failed complete run below.

Historical v0.1.45 results remain in the original revision-specific report. They are not counted as current-head passes.

Failed:

• The complete server run on v0.2.2 source 037314f9 executed 1,402 tests with two failures, one error and 21 skips. One failure was the corrected idle fixture ordering. The remaining failure/error reproduce on exact unmodified upstream v0.2.2 after adopt_user_defaults(): readable child identity and its ownership record fail. The same three selected lifeline tests pass on plain upstream and with only default-token DACL FA changed to GA. This is tracked separately in [Other] preserve Windows child identity after adopting user defaults #3399; the patch is not folded into this PR. The original complete run remains failed.
• Two preliminary merge-validation harness runs stopped at child-handle observation after upstream token adoption denied the required observer rights. Their owned jobs emptied; neither is counted as a pass. The final module-isolated checks and the narrower terminal cleanup scope are recorded separately in the report.

Not run:

• HIP, SYCL and Metal build/hardware qualification. The successful CUDA compile and CPU/protocol checks do not substitute for other backends or live GPU cache resizing under a model.
• A public-only real-model lease/compiler/reload workflow, arbitrary detached-tool containment, visible Desktop/media acceptance and complete normal-history continuation. Private combined-runtime results contain additional code and do not qualify this PR.
• A new complete server-suite pass after the fixture correction, or current-head CI results inferred from historical runs.

Limits

All touched-backend builds remain required before requesting review; this PR stays draft while those checks and scope review are unfinished. It targets a single engine; finite execution workspace and races with unrelated allocations remain. There is no universal no-OOM, uninterrupted-progress, foreground-performance or speedup claim. Prepared with Codex; contributor responsibility, authorship and license notices are retained.

[midhatn]

I extended this to cover the gap between an agent's model calls: under sustained memory pressure, Strata can now unload both the native and vision processes, then wait for full reload capacity before the next request or image encoding. This is opt-in and reuses the existing lifecycle/FIFO.

The final revision passed 886 Python tests (7 skipped) and five targeted native tests, plus the real idle-pressure Hermes cycle and HTTP vision/tool checks. The Hermes check verifies its final answer against a real four-job build, passing tests and a nonce available only in the tool output. Native/vision process replacement and cleanup were verified.

One important limitation of that test: a separate real 3 GiB allocation supplies pressure, and the tool response is deliberately held open. The build finishes before unloading. It demonstrates resource release and successful feedback across a tool wait, not faster compilation.

Review also caught image-download/admission races and missing guards after canceled active parking or timed/manual unload; those now have regression tests. The Windows reload estimate retains working-set peaks, and stale unloaded counters are hidden, crediting the specific reporting idea from #1093. #726 remains the credited allocator foundation.

The next gap is loading a smaller resident profile while another application stays open. Static 32/20 GiB tests worked, but automatic profile selection is not implemented. Pacing still has no demonstrated foreground speed benefit; this remains a draft, not a no-OOM guarantee.

Final Hermes cycle: 190.442 s including the controlled wait/reload, 2,096 build cases plus 104 independent checks, with minimum sampled headroom of 5.623 GiB RAM and 294 MiB VRAM. The current branch includes v0.1.40.4. Results and limits.

[midhatn]

Updated this draft onto v0.1.41 and added the missing proactive tool handoff. Hermes can now declare an upcoming heavy build before it starts, rather than waiting for memory pressure. Only the supervisor grants tool profiles; sequential workers use one broker. The companion generic Hermes change is #135418.

The live IQ3_S test verified one child compiler execution after both native and vision processes had exited, lease renewal/release, guarded reload, and a final answer grounded in the real tool output. The known-correct fixture passed 2,096 checks plus 104 independent checks. Minimum observed native VRAM headroom was 292 MiB. A separate GPU component test allocated, verified and freed 512 MiB, then resumed inference; minimum native headroom was 340 MiB.

There is one disclosed agent deviation: the supervisor also read its worker log. It did not compile twice, but that extra command violated the strict test prompt, so I am not claiming perfect task compliance. Full unload also loses warm state; these are lifecycle/correctness results, not an end-to-end speedup claim.

Evidence and limits are recorded. Final fork CI passed Windows/Linux server, broker, browser and native CPU jobs. Keeping this a draft for review, especially the lifecycle and backend boundaries.

[midhatn]

Another sequential Hermes image-to-video run completed with both protected tool handoffs, inference continuing after each return, and confirmed cleanup. It needed one image generation and one video generation, with no retries. The request, tool-effect, session and resource checks passed for that exact local composition, which includes the separately integrated checkpoint bank; this does not qualify the public PR head alone.

A fresh user question against a separate copy of the completed session also passed. Hermes recovered the saved artifact identities, attempt counts, cleanup state and recorded limitations without dispatching tools or repeating media generation. That checks transcript continuation, not warm-cache restoration. This particular session did not trigger history compression.

The newer Strata release is being integrated and tested separately. I am keeping the validated composition intact until the replacement passes the affected native and media checks. The PR remains draft: a clean repeat on the final composition and visible Desktop qualification are still outstanding. These results do not establish arbitrary foreground coexistence or a universal no-OOM guarantee.

[midhatn]

Updated the draft to v0.1.45 in 667deb15, preserving the public feature history and resolving the merge conflicts. The separate checkpoint-bank implementation is still excluded.

The merge needed a correction for the optional merged prompt window: its earlier prompt predictions must not count as emitted tokens or trigger EOS. The new regression covers that case. On this public composition, 1,298 server tests passed (13 skipped), along with 54 broker, three setup, 12 browser-control and seven native CPU tests. Exact scope and results.

The local combined stack also has bounded compiler handoff and completed-result recovery evidence, but it includes additional components, so those results do not qualify this public head. Normal history compression and the visible Desktop workflow still need work. This stays draft; public CUDA/HIP/SYCL builds and model-level validation remain outstanding. No performance or universal no-OOM claim.

[Niko1221]

Thanks for the pull request.

We're resetting the tracker for 0.2.0-beta. There are more than 1,300 open issues and pull requests, too many for us to review properly, so we're closing everything that isn't being worked on right now. Closed here does not mean rejected.

To reopen it takes about two minutes:

1. Rebase on main if it no longer applies, and re-run your checks on 0.2.0-beta.
2. Start the title with its area: [Engine], [GUI], [TUI], [Benchmark], [Docs] or [Other].
3. Fill in the pull request template: the problem and result, and the validation (what you ran, on which revision, what you saw).
4. Comment /reopen. A check runs and either reopens the pull request or lists what is still missing.

The rules are in CONTRIBUTING.md.

[midhatn]

Thanks for explaining the reset. This is still active work. I am bringing the local integration forward to v0.2.1 / engine 0.1.46 and checking shutdown, request ownership and unload/reload behavior before updating the public branch.

The local native build and 19 selected CPU/CUDA component tests pass on Windows with a Ryzen 9 7940HS and RTX 4070 Laptop. Server integration checks are still in progress. These results belong to the local combined stack, which contains additional work; they do not qualify the current public head or establish a performance improvement.

I will update the public branch, title and validation template before requesting reopening. Keeping the existing thread avoids opening a duplicate while that qualification is unfinished.

[midhatn]

Progress update: the unpublished public-only candidate is now based on v0.2.2 (037314f9). This PR's published head remains 667deb15; the separate checkpoint-bank implementation is still excluded from the public candidate.

The candidate's full server run executed 1,402 tests with two failures, one error and 21 skips. I have classified all three unsuccessful cases:

• One idle-reservation fixture checked cleanup before the server's finally completed. A five-line bounded wait fixes that ordering; all 42 idle HTTP tests then passed.
• The other failure and error reproduce on unmodified v0.2.2 after adopt_user_defaults(). The isolated FA → GA correction makes all three selected lifeline tests pass; that independent Windows fix is tracked in [Other] preserve Windows child identity after adopting user defaults #3399.

The original full run remains failed; these focused results are not a replacement full-suite pass.

Separately, the private combined prototype completed one bounded Windows API/broker lifecycle at IQ3_S, 65,536 context and high reasoning: native and vision processes exited before a real C++ compiler ran, the compiler returned the expected result, capacity-gated reload completed, and the next model answer recalled both the tool result and the earlier value. All 14 retained process handles signaled at final cleanup. The harness supplied the tool-call envelope, so this demonstrates the component handoff, not autonomous Hermes task selection or the visible Desktop workflow. It also does not qualify this public PR source.

Next is exact-source native qualification of the public candidate, followed by updating the branch and validation template. Public CUDA qualification, HIP/SYCL builds and the remaining workflow acceptance are still outstanding. This stays a closed draft while that work proceeds; no performance claim.

[midhatn]

/reopen

Updated the public branch to the v0.2.2 composition at 8f99733c, added the area-prefixed title and filled the validation template with exact tested revisions. The full server run remains recorded as failed; the focused idle-fixture pass and baseline DACL classification are separate results. This remains a draft with native/all-backend and public runtime qualification gaps listed. This supersedes my earlier statement that the draft would remain closed while that work proceeds; reopening is not a request to mark it ready for review.

[github-actions]

Reopened, thank you. A maintainer will pick it up from here. If it conflicts with main, rebase it first; that speeds up review.

[midhatn]

Merged current upstream v0.2.3.1 (7ddc17a3) while preserving the branch history, and completed exact-source Windows CUDA qualification on a69684f1.

• The CUDA 13.4 / MSVC 14.44 / architecture 89 portable build completed all 183 steps. All 13 native utility, protocol and parity test programs passed; live_memory_test ran with --protocol-only.
• Configure, build, every test and the outer worker exited zero. All 1,638 retained process handles signaled exit and owned-process cleanup was verified. The produced strata.exe SHA-256 is 744205e4e024b5cbba19e4b560678ffe21e6899e59050929f7af51557f3c6198.
• The merge's focused server/web checks passed 187 of 188 collected tests, with one optional jsonschema skip. The separate terminal drafting check passed 40 tests. The earlier complete-suite failure remains recorded and classified, including the independent Windows token-default issue tracked in [Other] preserve Windows child identity after adopting user defaults #3399.

Report-only commit b580e627 records the exact source, configuration, native tests, identities and remaining checks in bench/results/2026-10-23-adaptive-022.

This remains a draft. No model was started during native qualification. HIP/SYCL/Metal and the public-only real-model lease/compiler/reload workflow remain unqualified. The separate private checkpoint prototype and newer Python scheduling experiment are not included in these results.

[midhatn]

The ongoing work now has a canonical home in Adaptive Inference, with a release and an evidence ledger. New development lives there as a portable gateway with bounded pending-request scheduling, timing feedback and optional engine/agent integrations.

The new gateway itself passed 45 software tests plus four reconstruction checks in all six Windows/Linux/macOS Python 3.11/3.12 CI jobs. A separate ready-backend run through the gateway completed eight exact-integer model answers and engaged learned timing for both gateway classes. The separately operated backend exited all six retained processes. Its first attempt after idle unload timed out under reload_footprint_unavailable; that failure and the narrower ready-backend acceptance are retained. Gateway classes are consumed by its own queue, so this is not a new native-priority or lease test.

The repository also publishes hash-checked source overlays for the earlier Strata/Hermes composition. Its predecessor Windows owner completed eight correct real-model requests, used three prior timing samples on each class's fourth request, and confirmed all eight retained process handles exited. Those historical results did not execute the new gateway; that owner itself is not shipped by these overlays.

This PR remains the scoped upstream memory-relief/lease draft. The downstream scheduler, #3399 composition, source-reconstruction checks and new gateway must retain their own identities and validation. This update does not mark the draft ready, transfer downstream results to its public head, or claim a speedup or universal memory safety.

[midhatn]

Published 06ba725d8b9bb42b457302f71dd85204e9760c32 as a non-forced fast-forward from b580e627.

The coadaptive advisor's selected PCIe route now reaches the native request through strata_tune.pcie_frac; explicit nested routes, including zero, retain precedence. Other tuning fields and disabled/shadow behavior are preserved. The new Service-to-serializer regression fails on the preceding public head (three tests, four failed assertions/subtests); the corrected candidate passes all 46 focused routing/coadaptive tests:

python -B -m unittest serve.test_coadaptive_route_wire serve.test_coadaptive serve.test_coadaptive_reserve serve.test_routing_costs -v

These use the actual service/serializer with a fake engine. No new native build or model run is claimed. Native source, #3399, and the separate pending scheduler/latency-learner experiment are unchanged by this update. The PR remains draft, and the earlier complete-suite failures and remaining backend/model qualification gaps remain explicit in its description.

The reusable external gateway and independently preserved source are available in Adaptive Inference v0.2.0. Its evidence does not substitute for this PR's own qualification.