Skip to content

WIP: fix object lifetime and cleanup regressions - #1816

Draft
fglock wants to merge 318 commits into
masterfrom
wip/object-lifetime-ownership-20261015-124922
Draft

fglock wants to merge 318 commits into
masterfrom
wip/object-lifetime-ownership-20261015-124922

Conversation

@fglock

@fglock fglock commented Oct 15, 2026 •

Copy link
Copy Markdown
Owner

Summary

  • Fix the Mojolicious controller and WebSocket lifetimes regress with #1642 integration #1794 Mojolicious controller and WebSocket lifetime regressions and the Catalyst upload tempfile is deleted during request in #1642 integration #1795 Catalyst upload tempfile lifetime regression, with project-owned regression coverage.
  • Preserve callback captures while Perl containers own callbacks and release captured request graphs after their final owner disappears.
  • Track direct hash-entry and array-element owner slots, including replacement, aliasing, and removal paths; strengthen lifecycle root proofs and scope-exit cleanup.
  • Synchronize scalar container-owner metadata accessed from concurrent Perl threads.
  • Implement writable Internals::SvREFCNT compatibility, with a unit regression test validated against blead Perl and both PerlOnJava backends.
  • Add hash- and array-element scalar-reference regressions for exact Perl-visible owner-count transitions (1 → 2 → 1), final release, weak clearing, and exactly-once DESTROY.
  • Add true refaliasing coverage proving aliases to aggregate cells do not add an extra referent owner.
  • Follow typed hash/array owner slots toward Perl roots before using the whole-root reachability fallback, including DESTROY resurrection checks; Java regressions verify nested owner chains require zero root queries and snapshots.
  • Retain scalar-reference contents when copied into array/hash elements and release them when the final owner disappears; add a seven-assertion standard-Perl regression.
  • Cover dynamic localization of returned scalar references, including exact B::svref_2object count and final destruction on restoration.
  • Advance the Rewrite reference counting with complete Perl-compatible ownership semantics #1649 ownership rewrite through aggregate-slot recovery. The broader rewrite remains incomplete: scalar/proxy ownership migration and removal of cleanup reachability fallbacks remain next steps.
  • make ci creates the Windows runner's root-relative tmp directory for existing tests that use POSIX /tmp paths and uses four Gradle workers on Windows to finish the full shard set within the step budget. Unix worker defaults remain two.
  • Keep this PR in draft while Phase 2 continues.

Current progress (2026-10-23)

  • The ownership branch includes the careful merge of origin/master through aeda28344, preserving the Catalyst Catalyst upload tempfile is deleted during request in #1642 integration #1795 fix and overlapping owner-slot reachability changes.
  • Plain-array and hash tails in nested list assignment now transfer through the destination's owner-aware store path. Displaced values are released, scalar and container slots are registered, and tied/autovivifying hashes use their normal assignment path.
  • Immutable code commit 85daddac0 passes nice -n 19 make in 17m20s. The array-tail and hash-tail Perl regressions each pass 4/4 on system Perl, JVM, and interpreter.
  • The hash-tail regression fails all four assertions on the unfixed parent on both backends. Controller lifetime, WebSocket server push, Catalyst upload tempfile lifetime, and Catalyst global-destruction cleanup regressions pass on JVM and interpreter; Catalyst files report 24/24 and 3/3 assertions respectively.
  • Hosted Linux and Windows checks pass on preceding head c1167ffbd. GitHub has not attached a check run to current PR head 993d5875e; this PR remains draft. Phase 2 is incomplete because lifecycle dispatch still uses RuntimeBase.refCount and owner-slot accounting is not yet the default across scalar/proxy and aggregate paths.

Validation

  • nice -n 19 make — passed on immutable code commit 85daddac0 in 17m20s. All unit shards, the WebSocket task, and Joni packaging passed.
  • MortalListAggregateOwnerSlotRootChainTest — passed in the full unit gate; nested hash owner slots restored the count with zero root queries and zero graph snapshots.
  • DestroyDispatchOwnerSlotRootChainTest — passed in the full unit gate; a nested hash owner proves resurrection with zero root queries and zero graph snapshots.
  • scalar_reference_return_owner_lifetime.t — 7/7 on blead Perl 5.45.4, JVM, and interpreter; the unfixed PerlOnJava candidate failed the aggregate-retention assertion on both backends.
  • scalar_reference_localized_owner_lifetime.t — 6/6 on blead Perl 5.45.4, JVM, and interpreter, including an exact refcount match during localization and exactly-once destruction after restoration.
  • hash_element_scalar_reference_lifetime.t — 3/3 on blead Perl 5.45.4, JVM, and interpreter.
  • hash_element_scalar_reference_refcount.t — 6/6 on blead Perl 5.45.4, JVM, and interpreter; exact 1 → 2 → 1 ownership counts matched Perl.
  • array_element_scalar_reference_refcount.t — 6/6 on blead Perl 5.45.4, JVM, and interpreter; exact counts and final release matched Perl.
  • aggregate_scalar_alias_refcount.t — 8/8 on blead Perl 5.45.4, JVM, and interpreter; array and hash refaliases preserve Perl counts and release once.
  • Historical GitHub Actions run 39552652526 for head 59d14d58c.
  • nice -n 19 make check-links — passed after the ownership design status update on 5550984a1.
  • dev/tools/tests/ci_workflow_contract.t — 23/23 passed on blead Perl 5.45.4.
  • make -n ci OS=Windows_NT — confirms Windows temp setup precedes the build and Gradle is configured for four workers.
  • GitHub Actions run 39513358260 passed: Windows make ci in 52m13s and the focused Windows thread gate; Ubuntu make ci, thread compatibility, and SBOM gates.
  • Follow-up run 39520815459 passed on head b3f963739: Windows make ci in 68m21s and the focused thread gate; Ubuntu build, thread compatibility, and SBOM gates.
  • internals_svrefcnt_write.t — 6/6 on blead Perl; the unfixed PerlOnJava parent failed all six assertions; fixed JVM and interpreter backends pass.
  • ordinary_aggregate_element_owner_refcount.t — 11/11 on blead Perl 5.45.4, JVM, and interpreter.
  • mojolicious_controller_destroy_after_finish.t — passed on JVM and interpreter in the post-fix focused runner; full Make also passed.
  • mojolicious_websocket_server_push_lifetime.t — passed on JVM and interpreter in the post-fix focused runner; the full Make WebSocket lifetime integration test passed.
  • catalyst_request_body_tempfile_lifetime.t — 24/24 on JVM and interpreter after the ownership fix.
  • refcount/issue_1794_controller_owner_release.t — 3/3 on blead Perl 5.45.4, JVM, and interpreter.
  • timeout 1800 nice -n 19 ./jcpan -t HTML::Tree — 23 files / 989 tests passed, including unchanged t/refloop.t (8/8).
  • catalyst_request_body_tempfile_global_destroy.t — 3/3 on JVM and interpreter after the ownership fix.
  • list_assignment_array_tail_lifetime.t — 4/4 on system Perl, JVM, and interpreter.
  • list_assignment_hash_tail_lifetime.t — 4/4 on system Perl, JVM, and interpreter; all four assertions fail on the unfixed JVM/interpreter parent.
  • nice -n 19 make check-links — passed after the progress and changelog updates.
  • Earlier validated thread gates: JVM virtual 1925/1925, interpreter virtual 1925/1925, and platform focused 485/485. These were run before the final Mojolicious controller and WebSocket lifetimes regress with #1642 integration #1794 interpreter cleanup changes.

Test plan

  • Validate Perl-level regressions with blead Perl 5.45.4.
  • Run the focused Mojolicious regressions on JVM and interpreter with loopback access.
  • Run the Catalyst tempfile regression on JVM and interpreter.
  • Validate writable Internals::SvREFCNT on blead Perl, JVM, and interpreter, including parent failure evidence.
  • Validate hash- and array-element scalar-reference refcount regressions with blead Perl and both backends.
  • Run full nice -n 19 make on immutable code commit ecc5efd37.
  • Validate true aggregate refaliasing counts and cleanup on blead Perl and both backends.
  • Run the HTML-Tree integration suite, including t/refloop.t.
  • Run make check-links after the design-doc update.
  • Confirm GitHub Actions on latest PR head 993d5875e.
  • Continue Phase 2 owner-slot migration and its acceptance tests.

@fglock
fglock force-pushed the wip/object-lifetime-ownership-20261015-124922 branch 6 times, most recently from 0ca8e24 to a190027 Compare October 17, 2026 18:38
fglock added a commit that referenced this pull request Oct 18, 2026
Record the passing local make and latest Linux and Windows CI evidence for
PR #1816, including the added weak-slot rescue regression. Keep the remaining
Catalyst destruction warnings and Phase 2 ownership boundary explicit.

Generated with [OpenAI Codex](https://openai.com/codex/)

Co-Authored-By: OpenAI Codex <codex@openai.com>
fglock added a commit that referenced this pull request Oct 18, 2026
Record that the WebSocket release regression is now part of draft PR #1816.

Generated with [OpenAI Codex](https://openai.com/codex)

Co-Authored-By: OpenAI Codex <chatgpt-codex@users.noreply.github.com>
@fglock
fglock force-pushed the wip/object-lifetime-ownership-20261015-124922 branch from de68ca5 to 781648e Compare October 20, 2026 01:21
fglock added a commit that referenced this pull request Oct 20, 2026
Record the passing local make and latest Linux and Windows CI evidence for
PR #1816, including the added weak-slot rescue regression. Keep the remaining
Catalyst destruction warnings and Phase 2 ownership boundary explicit.

Generated with [OpenAI Codex](https://openai.com/codex/)

Co-Authored-By: OpenAI Codex <codex@openai.com>
fglock added a commit that referenced this pull request Oct 20, 2026
Record that the WebSocket release regression is now part of draft PR #1816.

Generated with [OpenAI Codex](https://openai.com/codex)

Co-Authored-By: OpenAI Codex <chatgpt-codex@users.noreply.github.com>
@fglock
fglock force-pushed the wip/object-lifetime-ownership-20261015-124922 branch from 772d19e to 5667a35 Compare October 20, 2026 17:47
fglock added a commit that referenced this pull request Oct 20, 2026
Record the passing local make and latest Linux and Windows CI evidence for
PR #1816, including the added weak-slot rescue regression. Keep the remaining
Catalyst destruction warnings and Phase 2 ownership boundary explicit.

Generated with [OpenAI Codex](https://openai.com/codex/)

Co-Authored-By: OpenAI Codex <codex@openai.com>
fglock added a commit that referenced this pull request Oct 20, 2026
Record that the WebSocket release regression is now part of draft PR #1816.

Generated with [OpenAI Codex](https://openai.com/codex)

Co-Authored-By: OpenAI Codex <chatgpt-codex@users.noreply.github.com>
@fglock
fglock force-pushed the wip/object-lifetime-ownership-20261015-124922 branch from 4eebdf5 to 44c0ed9 Compare October 21, 2026 12:52
fglock added a commit that referenced this pull request Oct 21, 2026
Record the passing local make and latest Linux and Windows CI evidence for
PR #1816, including the added weak-slot rescue regression. Keep the remaining
Catalyst destruction warnings and Phase 2 ownership boundary explicit.

Generated with [OpenAI Codex](https://openai.com/codex/)

Co-Authored-By: OpenAI Codex <codex@openai.com>
fglock added a commit that referenced this pull request Oct 21, 2026
Record that the WebSocket release regression is now part of draft PR #1816.

Generated with [OpenAI Codex](https://openai.com/codex)

Co-Authored-By: OpenAI Codex <chatgpt-codex@users.noreply.github.com>
@fglock
fglock force-pushed the wip/object-lifetime-ownership-20261015-124922 branch from a0c1449 to 70e715d Compare October 21, 2026 17:59
fglock added a commit that referenced this pull request Oct 21, 2026
Record the passing local make and latest Linux and Windows CI evidence for
PR #1816, including the added weak-slot rescue regression. Keep the remaining
Catalyst destruction warnings and Phase 2 ownership boundary explicit.

Generated with [OpenAI Codex](https://openai.com/codex/)

Co-Authored-By: OpenAI Codex <codex@openai.com>
fglock added a commit that referenced this pull request Oct 21, 2026
Record that the WebSocket release regression is now part of draft PR #1816.

Generated with [OpenAI Codex](https://openai.com/codex)

Co-Authored-By: OpenAI Codex <chatgpt-codex@users.noreply.github.com>
@fglock
fglock force-pushed the wip/object-lifetime-ownership-20261015-124922 branch from 565e0bd to bd3e3d6 Compare October 21, 2026 23:51
fglock added a commit that referenced this pull request Oct 21, 2026
Record the passing local make and latest Linux and Windows CI evidence for
PR #1816, including the added weak-slot rescue regression. Keep the remaining
Catalyst destruction warnings and Phase 2 ownership boundary explicit.

Generated with [OpenAI Codex](https://openai.com/codex/)

Co-Authored-By: OpenAI Codex <codex@openai.com>
fglock added a commit that referenced this pull request Oct 21, 2026
Record that the WebSocket release regression is now part of draft PR #1816.

Generated with [OpenAI Codex](https://openai.com/codex)

Co-Authored-By: OpenAI Codex <chatgpt-codex@users.noreply.github.com>
@fglock
fglock force-pushed the wip/object-lifetime-ownership-20261015-124922 branch from 2810e89 to d0ca160 Compare October 22, 2026 15:47
fglock added a commit that referenced this pull request Oct 22, 2026
Record the passing local make and latest Linux and Windows CI evidence for
PR #1816, including the added weak-slot rescue regression. Keep the remaining
Catalyst destruction warnings and Phase 2 ownership boundary explicit.

Generated with [OpenAI Codex](https://openai.com/codex/)

Co-Authored-By: OpenAI Codex <codex@openai.com>
fglock added a commit that referenced this pull request Oct 22, 2026
Record that the WebSocket release regression is now part of draft PR #1816.

Generated with [OpenAI Codex](https://openai.com/codex)

Co-Authored-By: OpenAI Codex <chatgpt-codex@users.noreply.github.com>
fglock and others added 28 commits October 25, 2026 04:26
Move global CODE owner release ahead of the generic undefineCodeReference
return, which previously bypassed the slot cleanup path. Keep saved coderef
aliases observing the undefined CV and retain the legacy bridge fallback.

Generated with OpenAI Codex (https://openai.com/codex)
Co-Authored-By: OpenAI Codex <codex@openai.com>
Restore localized CODE refs through their scalar owner slot when present,
retiring the slot with the count before lifecycle dispatch. Keep the legacy
count bridge for older cells and add a focused restoration regression.

Generated with OpenAI Codex (https://openai.com/codex)
Co-Authored-By: OpenAI Codex <codex@openai.com>
Document that current hash and array slots provide reachability identity while
scalar cells still carry counted ownership. Record the coordinated store and
release work required before aggregate recovery can be removed.

Generated with OpenAI Codex (https://openai.com/codex)
Co-Authored-By: OpenAI Codex <codex@openai.com>
Defer the tracked reference and I/O owner release after removing a non-final
array element, preserving the delete result until callers can retain it.
Add a regression for DESTROY timing across both execution backends and system
Perl, and document the compatibility fix.

Generated with Codex (https://openai.com/codex)
Co-Authored-By: Codex <codex@openai.com>
Update the ownership migration notes with the confirmed middle-array-delete
lifetime bug, regression evidence, and the still-open recovery callers.

Generated with Codex (https://openai.com/codex)
Co-Authored-By: Codex <codex@openai.com>
Record the issue-specific controller, WebSocket, and Catalyst tempfile tests
passing on both execution backends after the array owner-release fix.

Generated with Codex (https://openai.com/codex)
Co-Authored-By: Codex <codex@openai.com>
Only defer the deleted scalar's owner release when the array owns that
particular element. This keeps borrowed @_ aliases from releasing a caller's
reference while retaining cleanup for ordinary array elements.

Add a system-Perl-validated regression for DESTROY timing and update the
ownership notes and changelog. Full nice -n 19 make passes.

Generated with Codex (https://openai.com/codex)
Co-Authored-By: Codex <codex@openai.com>
Remember whether a localized array element was borrowed or owned and restore
that classification after reinstating the slot. Otherwise restoring an @_ alias
marks it as owned and argument-frame cleanup releases the caller's reference.

Add a DESTROY timing regression verified on system Perl and both backends.
Update the ownership migration notes and changelog.

Generated with Codex (https://openai.com/codex)
Co-Authored-By: Codex <codex@openai.com>
Verify that delete local preserves a genuinely owning array slot after its
returned value is dropped, and that clearing the restored slot releases the
object. Record the system Perl, JVM, interpreter, and full-gate results.

Generated with Codex (https://openai.com/codex)
Co-Authored-By: Codex <codex@openai.com>
Keep the scalar cell's existing value owner while a strong scalar refalias
survives hash or array clear and deletion. Release that owner when the final
alias disappears, without adding a duplicate referent count. Weak aliases and
borrowed array slots retain their existing cleanup behavior.

Add an 18-assertion regression validated with system Perl, JVM, and interpreter.
Update the ownership progress notes and changelog. Full `nice -n 19 make`
passes in 15m57s.

Generated with Codex (https://openai.com/codex)
Co-Authored-By: Codex <codex@openai.com>
Record which Perl owner edges use counted slots and which remain identity
proofs for aggregate recovery. Keep the scan until a generic deferred-release
producer is tied to a reproducible Perl-visible mismatch.

Generated with Codex (https://openai.com/codex)

Co-Authored-By: Codex <codex@openai.com>
Record the acquisition and release pairing for every generic deferred
referent decrement and narrow the remaining recovery audit to zero transitions
at the two production scan callers.

Generated with Codex (https://openai.com/codex)

Co-Authored-By: Codex <codex@openai.com>
Aggregate recovery already counted rooted hash cells, scalar values, and tie
wrappers, but omitted distinct reachable scalar-reference contents cells. When
another rooted aggregate owner made recovery succeed, the scalar overwrite
fallback could not restore the omitted contents owner. Count reachable
contents cells once per underlying cell while excluding weak and
non-counting aliases, and cover the mixed-owner recovery state.

Generated with Codex (https://openai.com/codex)

Co-Authored-By: Codex <codex@openai.com>
Preserve the pending ownership design update before continuing the Phase 2
owner-slot migration.

Generated with [Codex](https://openai.com/codex)

Co-Authored-By: Codex <158243242+openai-codex[bot]@users.noreply.github.com>
Route thread-cloned scalar references and blessed glob aliases through the
SCALAR_VALUE owner-slot API so their legacy count and identity ledger move
together. Add focused release assertions for both paths.

Generated with [Codex](https://openai.com/codex)

Co-Authored-By: Codex <158243242+openai-codex[bot]@users.noreply.github.com>
Document the cloned-scalar and blessed-glob owner acquisitions now routed
through SCALAR_VALUE slots, their verification, and the remaining recovery
boundary.

Generated with [Codex](https://openai.com/codex)

Co-Authored-By: Codex <158243242+openai-codex[bot]@users.noreply.github.com>
Promote wrapper owner slots during untracked-to-tracked handler transitions
when their owning tied container is rooted. This accounts for the wrapper
edge directly instead of relying on aggregate recovery to restore it.

Generated with [Codex](https://openai.com/codex)

Co-Authored-By: Codex <158243242+openai-codex[bot]@users.noreply.github.com>
Document the promotion path for rooted tie wrappers and retain the remaining
recovery callers as open Phase 2 work.

Generated with [Codex](https://openai.com/codex)

Co-Authored-By: Codex <158243242+openai-codex[bot]@users.noreply.github.com>
Add opt-in, class-filtered diagnostics for the two aggregate owner recovery
paths. Record the pending release source and the rooted slot kinds used to
reconstruct an owner count, so the remaining recovery scan can be audited
against the controller and tempfile lifetime regressions.

Generated with [OpenAI Codex](https://openai.com/codex)

Co-Authored-By: OpenAI Codex <chatgpt.com>
Record extended-access controller, WebSocket, Catalyst global-destruction,
and focused ownership results across both backends. Correct the earlier
File::Temp trace note with the observed weak-walker and rooted-slot mismatch,
which still requires owner-container tracing before recovery changes.

Generated with [OpenAI Codex](https://openai.com/codex)

Co-Authored-By: OpenAI Codex <chatgpt.com>
Extend class-filtered aggregate recovery diagnostics with the owner container,
scalar cell, value type, ownership flags, and destruction state for each slot.
This allows the File::Temp weak-walker miss to be mapped to its actual Perl
owner cells without changing the default runtime path.

Generated with [OpenAI Codex](https://openai.com/codex)

Co-Authored-By: OpenAI Codex <chatgpt.com>
Record that the recovered File::Temp count is backed by two owned glob
reference cells with active scalar-value slots. The remaining mismatch is
between weak-walker reachability and these rooted aggregate owner edges.

Generated with [OpenAI Codex](https://openai.com/codex)

Co-Authored-By: OpenAI Codex <chatgpt.com>
Extend the opt-in recovery diagnostics with sweep-live membership, root flags,
and immediate parent owner slots for each recovery container. This identifies
which root proof accepts an aggregate that the weak sweep does not reach.

Generated with [OpenAI Codex](https://openai.com/codex)

Co-Authored-By: Codex <158243242+openai-codex[bot]@users.noreply.github.com>
Add diagnostic reachability query counters to the filtered owner recovery
trace so conservative budget exhaustion can be distinguished from a proven
root path.

Generated with [OpenAI Codex](https://openai.com/codex)

Co-Authored-By: Codex <158243242+openai-codex[bot]@users.noreply.github.com>
Expose whether a filtered destruction owner root query returned through
its conservative node or edge budget. This distinguishes a discovered root
path from an incomplete reachability proof in the diagnostic trace.

Generated with [OpenAI Codex](https://openai.com/codex)

Co-Authored-By: Codex <158243242+openai-codex[bot]@users.noreply.github.com>
Add a test-only environment filter that skips the owner recovery scan for
named Perl classes. The default remains unchanged; focused lifetime tests can
now expose ownership paths that still depend on the migration bridge.

Generated with [OpenAI Codex](https://openai.com/codex)

Co-Authored-By: Codex <158243242+openai-codex[bot]@users.noreply.github.com>
Record that the focused #1794/#1795, aggregate alias, and DESTROY regressions
pass with their class-filtered recovery scan bypassed. Keep the recovery scan
because synthetic recovery tests and both production callers remain. Remove
the temporary bypass switch after the audit.

Generated with [OpenAI Codex](https://openai.com/codex)

Co-Authored-By: Codex <158243242+openai-codex[bot]@users.noreply.github.com>
Record the uncapped negative aggregate root proof, successful class-filtered
recovery-bypass regressions, and the remaining source audit target. Refresh the
PR head and local gate status while keeping Phase 2 open.

Generated with [OpenAI Codex](https://openai.com/codex)

Co-Authored-By: Codex <158243242+openai-codex[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant