Skip to content

feat: add cooperative cancellation requests and claim-bound delivery - #291

Draft
rmcdaniel wants to merge 33 commits into
mainfrom
feat/cooperative-cancellation-service
Draft

rmcdaniel wants to merge 33 commits into
mainfrom
feat/cooperative-cancellation-service

Conversation

@rmcdaniel

@rmcdaniel rmcdaniel commented Sep 30, 2026 •

Copy link
Copy Markdown
Member

Customer outcome

Expose cooperative cancellation, prepared local execution and one coherent cascade diagnostic for shared issue 136. Preserve the original request identity, delivery and deadline through worker replacement. Keep this PR a draft until the complete model and exact published mixed-language cascade are qualified.

Implementation

  • Discover capabilities from the actual bound Native bridge. Default protocol remains 1.19. Cooperation, prepared locals and complete atomic all groups are explicit, unfrozen 1.20 Source opt-ins.
  • Validate canonical requests, delivery, original claim/owner/attempt authority and duplicate receipts. Observe cancellation independently of application heartbeats and lease renewal.
  • Expose bounded local/group checkpoints, preparation, recovery, control, heartbeat, outcome and stop acknowledgments. Admit complete immutable batches before callbacks run and distinguish fences from matching physical-stop reports.
  • Preserve encoded arguments, authored paths, quotas, payload continuation authority, original cleanup budgets and acknowledged child waits through cold replay.
  • Admit explicit remote Activity policies only through the capable original claim and installed backend, with concrete capability diagnostics. Remote Abandon requires a finite total lifetime. Prepared local TryCancel/WaitCancellationCompleted require a separately capable issued claim and the actual installed backend policy list. Local Abandon remains refused.
  • Hold detached remote Activity history and external payloads until canonical closure, including safe fallback for older installed Native.
  • Add Native's shared bounded, namespace-scoped cancellation cascade view to current-run and exact-run diagnostics. Distinguish installed-runtime support from absence of a request. Preserve original root budget, historical run selection, stop receipts and recorded recovery evidence. Generic findings distinguish historical task recovery from active faults.

Current exact-head qualification

Server dd8996fdd6d488b48d575b1b934821bd8bedc8ed binds explicit prepared-local policies to prepared_local_activity_cancellation_policies on the original issued workflow claim. The capability requires cooperation, prepared-local support and protocol 1.20. Discovery uses the actual installed optional policy list. Admission checks precede payload resolution and partial group writes, and are repeated under mutation authority. Replacement polling checks canonical Scheduled policy before probing and under the run lock. Omission preserves legacy admission/replacement. Re-registration cannot upgrade old claims. Default protocol 1.19 and older optional-role bridges advertise no explicit local policies.

Local new HTTP cases pass 18 / 240, the complete Source lane 122 / 5905, affected regressions 311 / 3376, and OpenAPI/workflow/evolution contracts 27 / 264, all with zero errors, failures or skips. PHP 8.4.26, locked Laravel 13.34.0, isolated SQLite. Confirming scoped Pint and whitespace checks pass. Tracked Composer lock remains unchanged. Three-database Source 37079106466, binding qualified Native ba1aa4770c4b94533548bd45bd3ba028229c54e8, passes 122 / 5905 on each of SQLite, MySQL and PostgreSQL, including all 18 new cases without skips. Affected SQLite 311 / 3376 passes. Ordinary locked-package CI 37079089323 retains published Native 2.3.3 and passes 2387 / 48736 with 50 existing skips, six existing deprecations and zero errors/failures. Contracts, query HTTP, performance bounded-growth/smoke and public boundaries pass. Raw Source provenance and results are retained until December 31, 2026. Physical SDK qualification remains separate.

Preceding historical task diagnostics qualification

Server 172dedb57a71629048fc145175d21eb626bf386d corrects the generic historical repair warning found in the final mixed cascade response. The existing Task History badge now produces an informational task_recovery_history finding with a useful explanation. Active/unclassified task problems retain warnings, and active replay failure retains its error. Task/failure records and cascade evidence are preserved.

Two real debug API cases reproduce the earlier warning on completed and cancelled runs. The complete affected file passes 15 cases / 200 assertions, zero errors/failures/skips, against locked published Native 2.3.3. Scoped Pint/whitespace checks pass and the tracked Composer lock is unchanged.

Source 37074355176, binding Native 47d3ade12f8f51bbd75cbddf4aa714040f1a4195, passes 104 cases / 5665 assertions on each of SQLite, MySQL and PostgreSQL. The three new diagnostic cases pass 35 assertions on each database. Affected SQLite regressions pass 311 / 3376. These selections have zero errors, failures or skips. Ordinary locked-package CI 37074348861 passes 2387 / 48674, with 50 existing skips, six existing deprecations and zero errors/failures. Repository contracts, query HTTP, performance and public-boundary gates pass. Raw Source results and provenance are retained until December 31, 2026.

Preceding cascade API qualification

Server 70807856b416b8792099108712cdd031e2c66c89 with Native 88ea6e46499332427d0f9a439768ae0bf86e8702 passes three-database Source CI and ordinary locked-package CI. Both conclusions are success.

Hosted Source selection passes 101 tests / 5630 assertions on each of SQLite, MySQL and PostgreSQL. All six new cascade API cases pass 122 assertions on each database. Affected SQLite regressions pass 311 / 3376. Candidate suites have zero errors, failures or skips. Ordinary feature CI retains published Native 2.3.3 and passes 2384 / 48635, with 50 existing skips and six existing deprecations. Raw Source evidence is retained until December 31, 2026. Affected local Pint and API checks also pass. Native's full matrix qualifies the exact shared reader dependency.

The preceding Server qualification names its exact older tuple. The real mixed Source recovery case passed with Server 8940fb8a866e24d7874947e850f7fc864fbdfd3c and Native 9f8cb61947997c1c60f294bdf559d02d8258cc18. Neither is proof of this new API or of published artifacts.

Required before publication

Finish local policy SDK consumers, settle remaining scope decisions and fair competitive contracts, and qualify the exact published cascade/UI. The updated full mixed Source scenario passes against preceding Server 708078, preceding Native 47d3ade and the exact SDK/CLI heads recorded there. CLI/Waterline inspection components retain separate qualified evidence. Verify stale publication refusal, unchanged delivery and duplicate identity through real cleanup-worker SIGKILL, all runs Cancelled before the original 30-second deadline and one coherent operator view. Keep shared issue 136 open until that outcome and fair competitive evidence are complete. Rust's breaking SDK source API changes require the separate human major-release decision before Rust publication.

@rmcdaniel

Copy link
Copy Markdown
Member Author

Connected Python/Server source qualification now passes eleven SQLite cases at Python 119747bb2fb838f6dff64c9938c780414a074b99 and Server 2e2d6b31df981e98c1a054a8953e9efe95d17e6e. Scenario evidence covers actual SIGKILL/cold process replacement, shutdown grace and expiry, original request/deadline and one canonical marker, waiting timer, local async/synchronous execution, remote heartbeat/result/failure fencing, deadline, termination and a discarded successful delivery acknowledgment.

The current default remains protocol 1.19. Explicit source candidate protocol 1.20 now discovers the capability through the same policy used for request admission. Python's normal suite passes 1,618 local tests, Ruff and strict mypy. Server's affected filter passes 145 tests and 5,165 assertions, with Pint passing.

Normal Python CI, Server CI and an explicit exact-pair MySQL qualification are running. The candidate run's three supported Python, package, corpus and lint jobs pass. It retains connected JUnit evidence and removes its stack.

Next: finish those source gates, then PHP/Rust equivalents, coordinated protocol/specification activation and the exact published service tuple. These source drafts do not authorize a published cooperative capability claim. Completed local task resources are being removed.

@rmcdaniel

Copy link
Copy Markdown
Member Author

Connected Python qualification complete

At Python 119747bb2fb838f6dff64c9938c780414a074b99 and Server 2e2d6b31df981e98c1a054a8953e9efe95d17e6e, normal Python CI, docs and boundaries pass. The explicit candidate MySQL run also passes every job. Its integration finished at 02:29:45 UTC on October 1, with 33 passed and one existing CLI-binary skip in 80.19 seconds. All eleven cooperative cases are present and passed in the downloaded JUnit artifact, retained through October 8. The log verifies the exact Server SHA and teardown passed.

Full scenario evidence covers original identity/deadline/one marker, a discarded successful delivery response, real SIGKILL and cold process replacement, local async/synchronous and remote fencing, shutdown grace/expiry, waiting timer, deadline and termination. Local SQLite passes all eleven cases. All local task containers, worktrees, dependencies, proofs and images are removed. Downloaded CI reports and transport files are removed after this handoff.

Server's source/corpus CI passes 2,279 tests and 47,800 assertions. MySQL replay/query topology passes. Polling bounded-growth smoke remains running. Both PRs remain drafts and default protocol remains 1.19.

Next: PHP/Rust request, delivery, canonical replay and shielded cleanup parity, remaining backend concurrency, coordinated specification/capability activation and the exact published Server/SDK tuple. No published cooperative capability is authorized by these source-only results.

@rmcdaniel

Copy link
Copy Markdown
Member Author

Remote activity observation qualified at 073a516bbd4063f57d4ad65ad732ec423131c8ff

The additive candidate1.20 POST /worker/activity-tasks/{taskId}/status is implemented and all normal Server gates pass at this exact head. Full feature/corpus and source qualification runs 2,300 tests / 47,999 assertions, with six existing PHPUnit deprecations. MySQL replay/query HTTP topology, polling bounded-growth smoke/performance qualification and both public boundaries pass. Local PHP8.4.26 passes 184 tests / 7,982 assertions, Pint on all four changed PHP files, worker OpenAPI evolution23→25 and git diff --check.

The21 new status cases cover exact namespace/task/attempt/owner fencing, stale attempts, canonical activity cancellation, terminal cancel/terminate, lease expiry and authored heartbeat/execution deadlines before timeout repair. They prove preservation of attempt, execution, task, worker registration, required session and history. Required sessions must remain active, owned and inside their lease/TTL. Missing/replaced/closed/expired sessions refuse continuation without repair. Backend pressure returns retryable503 with the attempted fence and no continuation grant. Observation remains available under draining/fenced storage admission.

This uses the existing published Workflow2.3.0 attempt observation primitive. It never renews an attempt lease, user heartbeat, registration or required session. The existing five-minute activity lease and authored user heartbeat renewal behavior remain. A pending cooperative request is not delivered cancellation. Publication must independently validate the attempt fence, and a prior observation is not a reservation.

The worker OpenAPI candidate advances to25. Default released protocol1.19 and terminal endpoint behavior remain unchanged. PHP PR91 is wiring the actual owning Worker to bounded observations and the existing process supervisor, with user heartbeats proxied separately. Actual blocked remote callbacks, shutdown, owner death and cold workflow replacement are the next connected qualification. This Server result does not by itself establish that SDK lifetime behavior. The draft remains gated by per-language parity and exact published-tuple conformance before capability activation or release.

@rmcdaniel

Copy link
Copy Markdown
Member Author

The cooperative Server candidate now consumes published Workflow 2.3.1 at fb3f3e59a4342fdebf8ced6160798906c3ee4387. Exact Server head is c32434cc784a1bf870dcf19568bef944a320f6a7. It merges the reviewed Server #292 dependency/release-metadata change into the previous 073a516bbd4063f57d4ad65ad732ec423131c8ff candidate. The candidate's service implementation is unchanged and protocol defaults remain 1.19. Normal source checks have restarted for this exact tuple.

Workflow's complete source matrix and all 16 published Laravel/PHP upgrade combinations pass. Server #292 is separately qualifying stable image 2.4.35. A focused ordinary-protocol signal drill reproduces the expired-workflow/older-active-activity defect on published Server 2.4.34 and PHP SDK 2.1.6, so this repair also affects existing service callers.

The entire Python candidate CI is dispatched at unchanged Python 88f0e31bf1736271deaabcc67d74df9ce98df491, with server_commit=c32434cc784a1bf870dcf19568bef944a320f6a7 and cooperative_qualification=true. It must pass the actual remote-worker supervision and SIGKILL cases using the published native correction. This is source-candidate qualification, separate from Server #292's published-image PHP/Python/Rust follow-through and shared #136's activation/reclaim/Rust gates.

@rmcdaniel

Copy link
Copy Markdown
Member Author

Heartbeat ownership interleaving reproduced and fixed at source

The ordinary protocol 1.19 regression fails against unchanged main d0692b171a0cabe6eefdc8bfc0f82d249e16128d with locked published Native Workflow 2.3.1. Its two HTTP kernels perform actual Native status, claim and renewal. The fixture only provides an IPC barrier and clocks crossing the one-second lease expiry. It does not edit lease rows or fabricate Native results.

Observed baseline:

{"heartbeat":{"workflow_task_attempt":1,"lease_owner":"original","renewed":true},"replacement_claim":{"workflow_task_attempt":2,"lease_owner":"replacement"}}

The same Native task ID was present in both responses. The positive heartbeat therefore identified a claim that no longer owned the task.

The fix in #294 head 226025ffe3e0c1af98a77c01932a50f41799d8d8 holds the namespace-scoped task lock across ownership validation and Native renewal. Local Pint and 79 focused protocol, ownership/error, success and poll-pressure cases pass with 4,286 assertions. The controlled race now retains the original durable owner/attempt and the replacement poll returns no task. Command: php vendor/bin/phpunit --filter 'WorkflowTaskHeartbeatRaceTest|WorkerProtocolOwnershipErrorContractTest|WorkerProtocolSuccessContractTest|SqliteWorkerPollLockPressureTest|WorkerPollBackpressureTest'.

Cooperative cancellation draft #291 carries the same locked renewal, plus the pending observation read inside that transaction, at 6be6cd39 (full exact revision available in its CI). Its 24 focused cancellation protocol and concurrency cases pass locally with 545 assertions, preserving the original request/deadline and one canonical request without inventing delivery.

Normal CI is running for both heads. This source result does not claim a published image or Cloud deployment. Next finish exact-head CI/review, publish Server 2.4.36/Helm 0.1.132 independently of cooperative cancellation and verify the affected published worker cells. Rust coordinator qualification then uses the requalified #291 head. Default protocol 1.19 and ordinary capabilities remain unchanged.

@rmcdaniel

Copy link
Copy Markdown
Member Author

Published Server 2.4.35 reproduces the ownership defect

The same committed regression ran inside the unchanged published Server image durableworkflow/server@sha256:49560f7f861271931125348a9d01638cd722e13ee976b5b732ef122548f15dab with its own PHP 8.3.35, Laravel and Native runtime code. PHPUnit 11.5.55 and Mockery 1.6.12 were mounted as separate test tools, together with the repository's tests. No application or vendor file was replaced. Reflection receipts verify that the controller and Native bridge load from the artifact, and both artifact provenance records name Native 2.3.1 at fb3f3e59a4342fdebf8ced6160798906c3ee4387.

The two actual HTTP kernels reproduce:

{"heartbeat_status":200,"heartbeat":{"workflow_task_attempt":1,"lease_owner":"original","renewed":true,"reason":null},"replacement_claim":{"workflow_task_attempt":2,"lease_owner":"replacement"}}

Both responses identify the same actual Native task. The unchanged published image therefore fails the ownership assertion, with 16 assertions reached. SQLite clocks and an IPC barrier control the interleaving; the harness does not edit any lease, owner or attempt row.

The original full CI result exposed an overly strict test assertion, not a valid positive acknowledgment. PHP 8.3 uses deferred SQLite transactions even when transaction_mode is configured, allowing replacement to win. The fixed transaction retries and returns a correct owner-mismatch refusal. The regression now requires either a positive acknowledgment matching the durable original claim or that exact refusal together with proof of the replacement claim. The same change preserves the cancellation request and cleanup deadline in #291. This still fails the old published image.

Next complete revised-head CI and review, publish Server 2.4.36, repeat this exact-image regression and verify the published PHP/Python/Rust lifecycle tuple before closing #293.

…ellation-service

# Conflicts:
#	app/Http/Controllers/Api/WorkerController.php
@rmcdaniel

Copy link
Copy Markdown
Member Author

The ordinary-protocol heartbeat ownership fix has shipped independently as Server 2.4.36, and #293 now has exact published before/after regression and published PHP/Python/Rust lifecycle evidence.

This draft incorporates merged main at 898c51375ddcb2c25588eb63e41fed26a3fd61b2. Current draft head is 24b54c9f8b3f84cad93ec58d5d7cd2f95160458d. Ownership validation, renewal and the candidate cancellation delivery read share the locked transaction. Default protocol 1.19 and the explicit candidate protocol 1.20 boundary remain unchanged.

Local formatting and the focused ownership/cancellation cases pass 25 cases and 565 assertions. Exact-head feature CI passes 2,302 cases and 48,041 assertions. Boundary and replay/query checks pass. Polling smoke is still running at this handoff, so no completed all-checks claim is made yet.

Next product action remains Rust #55's real claim/heartbeat cancellation carrier, canonical delivery refresh and replay coordinator, including callback ownership, replacement and physical lifetime checks. Candidate Server/Native and per-language connected gates must use the updated exact source tuple before the final published tuple is released. This stable heartbeat patch does not qualify or activate cooperative cancellation.

@rmcdaniel

Copy link
Copy Markdown
Member Author

The updated Server cancellation draft head 24b54c9f8b3f84cad93ec58d5d7cd2f95160458d now passes every normal exact-head check. Feature CI passes 2,302 cases and 48,041 assertions, and the formerly pending polling smoke passes. This source includes the independently released Server 2.4.36 ownership fence.

Rust #55 has advanced to 936303ddcea57789daf76889f5944d243758d7ce, with actual immutable claim/observation capture and bounded fenced history loading. All normal Rust exact-head CI passes, with 303 library cases and 24 integration/consumer/corpus cases.

Next action remains connecting that carrier to the Rust delivery/canonical-refresh/replay coordinator, preserving earlier command prefixes and proving committed delivery before application cancellation. The connected language and physical callback lifetime/ownership/replacement gates remain required before cooperative release or activation. Published Native remains Workflow 2.3.1.

@rmcdaniel

Copy link
Copy Markdown
Member Author

Pending cancellation remains runnable after a command prefix

The real Rust Worker qualification found a missing Server successor. At Server 24b54c9f8b3f84cad93ec58d5d7cd2f95160458d, a worker correctly committed a side effect before its cancellation boundary. The run then stayed waiting, with the original request undelivered and its only workflow task completed. No successor existed. The other four connected scenarios passed.

Fixed in 555cf7ab4ed4e9a79b52475be76a6dfcf886dd70. Successful nonterminal completion by a cancellation-capable actual claim now ensures one workflow task remains available for an undelivered request. The check and creation share the existing fenced completion transaction and run lock. Existing ready/leased tasks are reused, delivered requests do not create successors, and terminal runs or expired cleanup authority cannot resume. The original request and deadline remain unchanged. Workflow remains the published 2.3.1 package at fb3f3e59a4342fdebf8ced6160798906c3ee4387.

Two regression cases cover request before and after claim, two prefix side effects, repeated completion rejection, a different successor owner, delivery at sequence 3, and no leftover runnable task after terminal cancellation.

Raw counterfactual on the preceding Server head:

test_prefix_completion_keeps_pending_cancellation_deliverable [before claim]
Failed asserting that actual size 0 matches expected size 1.
test_prefix_completion_keeps_pending_cancellation_deliverable [after claim]
Failed asserting that actual size 0 matches expected size 1.
Tests: 2, Assertions: 16, Failures: 2.

Focused protocol suite after the fix:

PHP 8.3.35 / PHPUnit 11.5.55
OK (23 tests, 496 assertions)

Local feature/Nexus/corpus/OpenAPI run:

Time: 03:24.539, Memory: 56.00 MB
Tests: 2304, Assertions: 47873, PHPUnit Deprecations: 6, Skipped: 9.

The local run omits nine external-service cases. Full Server CI supplies those services and is running. Exact-candidate connected qualification is also running for Rust, PHP, and Python.

Next: inspect those results, address any remaining actual Worker/recovery failures, then qualify the complete published tuple before activating the capability. This PR remains a draft. Default Worker protocol remains 1.19.

@rmcdaniel

rmcdaniel commented Oct 1, 2026 •

Copy link
Copy Markdown
Member Author

Current source qualification

Server 3c15bfb0f98e038febb657412f1ca97bbdb23ad8 retains the qualified successor-task fix for a pending cancellation after a prefix completion. Ordinary claims without the cooperative capability skip the extra task refresh.

All current normal gates passed:

Connected source checks use this exact Server commit and published Workflow 2.3.1:

Shared recovery and replay gates and complete published-tuple qualification remain open. This remains a draft. Published artifacts and ordinary Worker defaults remain unchanged.

@rmcdaniel

Copy link
Copy Markdown
Member Author

Prepared-local Server source qualification is now verified at d3dcc3b9ba5a4af4ac0ddfd18c06cc0a554977c0 with Native e557be5fa697774175dc0b9aceb1b3e7f43d89ad.

SQLite, MySQL and PostgreSQL each passed 28 tests / 2,396 assertions with zero errors, failures or skips. The SQLite affected regression group passed 228 tests / 2,722 assertions. The MySQL run exposed JSON key order affecting a persisted worker process identity. The correction keeps strict fields, types and values while ignoring object key order, with negative fencing checks on every database.

Source run and retained raw evidence.

Next: connect the SDK local supervisors to this admission, original-attempt control and same-claim history refresh. This draft remains protocol 1.20 source work, with default 1.19 unchanged. These fixtures do not prove physical callback termination or the published mixed-language +30-second cleanup/SIGKILL scenario.

@rmcdaniel

Copy link
Copy Markdown
Member Author

Prepared cleanup and application heartbeat foundation is now source-qualified.

  • Native 7459c51845d01a16c9af1a38cd10d7628641a2bb: required CI passes full ECS/PHPStan, corpus, unit, policy and MySQL feature gates. The new cleanup/heartbeat suite passes 19 / 142, with all six existing prepared-local groups passing. Raw MySQL JUnit.
  • Server 929b457ce89ffad1f0d11d517f81a9f7aa4854f0 bound to Native runtime 12c5f7c68b7f6f5dc9d60a74363b8f38ff1d95a3: 32 / 3,275 on each of SQLite, MySQL and PostgreSQL, plus 228 / 2,722 affected regressions. Run, raw provenance/JUnit/logs. Native's later commit changes only test assertion ordering and formatting, with identical runtime files.

Cleanup admission now requires canonical request/delivery proof and a later authored sequence. Runtime-owned identity and original deadline survive retry and cold recovery. Supervisor control renews both leases without application heartbeats or fixed deadline changes. Actual application heartbeats are separate, deadline-bounded writes that renew neither lease. Server also refuses stale-registration cleanup publication after cancellation is accepted.

Commands are retained in the source CI: Native's explicit V2PortableLocalActivityCleanupTest MySQL step and existing prepared-local groups, and Server's exact-source three-database PreparedLocalActivityProtocolTest / worker-fence job plus affected regression suite.

Next action: wire PHP prepared replay to explicit cleanup proof, distinguish acknowledged application heartbeat deadlines from fixed execution/root budgets, then activate the actual local supervisor. These fixtures qualify backend and HTTP authority. Physical callback stop and the exact published 30-second mixed-language cleanup/SIGKILL cascade remain required. Both PRs remain drafts, default protocol remains 1.19, and #136 remains open.

@rmcdaniel

rmcdaniel commented Oct 2, 2026 •

Copy link
Copy Markdown
Member Author

Prepared local payload draining qualification

Server source: e288b8f0f0e46cae91d706e99bb1f4fb56030da2.
Native source overlay: 7459c51845d01a16c9af1a38cd10d7628641a2bb.
PHP source consumer: 53d0be081422d12354a2cd7d4517ffd5054d3089.

The prepared completion schema is advertised only when explicit protocol 1.20 and the actual bound optional Native role support it. Checkpoint, preparation, recovery and outcome slots keep their stable operation identity and share one original workflow claim's byte and slot allowance with legacy completion uploads.

New uploads require the original issued worker registration, current workflow epoch and live authority. Outcome uploads additionally check the admitted local attempt and all fixed execution deadlines. Accepted cancellation fences old ordinary callback results. Cleanup remains bounded by the original root deadline. Uploads do not renew workflow/activity leases or record application heartbeats. An exact stored reference can be read after completion without rewriting objects, budgets or retention.

The API scenarios exercise external prefix, arguments, recovery and result bytes through successful workflow completion, and shielded external cleanup through Cancelled. Refusal cases cover stale registration, absent prepared capability, wrong namespace/owner/epoch, unknown attempt, default protocol, expired activity/root deadline and exhausted shared allowance.

The test-only follow-up normalizes JSON reference objects through the existing strict reference validator. MySQL can reorder object keys without changing their values or types. Assertions for unchanged object and budget rows remain strict.

Source qualification job passed 40 tests / 3,831 assertions on each of SQLite, MySQL and Postgres, with zero errors, failures or skips. The affected SQLite regression suite passed 303 / 3,195, also with zero errors, failures or skips. The job checked and retained the exact Server/Native source provenance.

Raw provenance, JUnit and logs are retained for 90 days. JUnit SHA-256: SQLite ff9bf1f6dee7a3665cc858369ed3a5d642858ae45daa9adc69f0751827d20fc4, MySQL 71969e8b17a71350622310ca3cad406c8ae6dde61e21ba35768044fa9568e341, Postgres 5ebbafd36414999690a8971575e506d6d782c449b5bfb587c8f449129c13555a, affected regressions b3c57fb6090a8cfb03d873fd16430a6d525be679c9fae5523cc0ad2210ebe84f.

Repository contracts, central action policy, current locked-package feature CI, replay/concurrent HTTP and public-boundary checks passed. The full source-dispatch run also passed its locked-package feature and source qualification gates. Performance smoke passed, including its source qualification gate. PHP consumer CI is fully passing, as recorded in SDK PR #91.

Worker OpenAPI specification version advances from 29 to 30. Default published protocol remains 1.19. This is source qualification. Physical prepared SDK callback supervision and the published mixed-language 30-second cleanup/SIGKILL cascade remain the next required integration.

@rmcdaniel

Copy link
Copy Markdown
Member Author

Atomic prepared local group admission is Source qualified at Server c1bda7f81323e9939ef433c7ebca66f62365cdfd, explicitly bound to Native 3a9e1d4b5c0bf0e6f35580f2031dc729791c5cc7.

Three-database Source CI passes 53 tests / 5,046 assertions on each of SQLite, MySQL and PostgreSQL, plus 303 tests / 3,207 assertions in the affected SQLite regression suite. All have zero errors, failures or skips. Raw provenance, JUnit and logs are retained for 90 days.

Locked-package feature CI, concurrent HTTP/replay, performance smoke and both public-boundary checks pass. Local prepared HTTP/OpenAPI/completion-context qualification passes 108 / 5,297, and formatting passes.

Coverage includes every child/local sibling scheduled before preparation, immutable duplicate receipts and total deadlines, original issued group capability, namespace isolation, partial/terminal group and closed-descriptor refusal, whole-batch quota rollback, replacement claim before the first local attempt, shielded multi-local cleanup with one canonical delivery and original budget, external-payload draining through one original claim allowance, storage fencing and typed search-history updates after group checkpointing.

This adds transport/admission behavior using the existing Avro and payload-reference validators. The wire codec is unchanged. Source OpenAPI 1.20 advertises the additional group role only when the actual installed backend supports it.

Native's current head b60330bc484f861e7d611debf84d5e7e9940e248 has identical runtime files to this qualified binding and passes full database/unit/coverage Source CI. Its later changes affect test snapshot comparisons only.

These are HTTP/backend fixtures. PHP's managed concurrent group consumer is in progress in sdk-php#91. The exact published PHP parent → Python child → Rust remote activity plus PHP local activity, physical stop, cleanup SIGKILL, replacement replay, original +30-second convergence and one cascade view still gate shared issue #136. This PR remains draft. No release or Cloud rollout has occurred.

@rmcdaniel

Copy link
Copy Markdown
Member Author

Current Native binding qualification

Server head remains c1bda7f81323e9939ef433c7ebca66f62365cdfd. Native 8fa63fd9d65dd9aed88adac4a267104760ff4b09 now fixes cancellation history group grammar and the successor-claim authority needed to resume interrupted siblings. The fix and local qualification are recorded on Native PR 603.

Both PHP connected group cases pass locally on this pair, with PHP 720810a0b9b062fe89690aabd1cae46313c97d71: physical stop without application heartbeats, group cleanup, owning-worker SIGKILL, both interrupted attempts recovered, unchanged canonical delivery and duplicate request identity/deadline, and Cancelled closure within the original 30-second budget.

Exact three-database Source qualification and exact connected PHP qualification are running. The published tuple and default protocol remain unchanged. This PR stays a draft, and shared issue 136 remains open for its full mixed-language customer outcome.

@rmcdaniel

Copy link
Copy Markdown
Member Author

Current prepared-local Source pair qualified

Source qualification 36996624733 passes all gates at Server c1bda7f81323e9939ef433c7ebca66f62365cdfd, using readonly Native 8fa63fd9d65dd9aed88adac4a267104760ff4b09.

Prepared-local protocol and poll-fence cases pass 53 tests / 5,046 assertions on each of SQLite, MySQL and PostgreSQL, with zero errors, failures or skips. Affected cancellation, worker, payload, schema and capability regressions pass 303 / 3,207, also with zero errors, failures or skips. Locked-package feature CI passes separately.

Raw database JUnit, logs and source provenance are retained for 90 days. JUnit SHA256:

  • SQLite: 4967655ab5d9b9fca00049e6b6cc751de86680939423706ff99b84f2f1047196
  • MySQL: 977e0c025ad7fadd76505e674fcf91a4476360e8ea20371cf64340387db09316
  • PostgreSQL: 90a910124973679a8e59e3ca2e353c8bd3793715db45ea85fb2c5a27c4a0df88
  • Affected regression: 3433f257f85fd56d64385ed352ef4418916fe98c3a3839142a537b032a2a578c

Native head 850a64050b57d47181abeae86d163962c9047317 changes only test formatting from the bound revision. src/ and resources/ are byte-identical. Its full Source run remains in progress. Exact connected PHP group and cancellation qualification is also in progress. Published artifacts remain unchanged. This PR remains a draft, and shared issue 136 stays open for the complete published mixed-language cancellation model.

@rmcdaniel

Copy link
Copy Markdown
Member Author

Detached Activity retention Source qualification

Server 0f02ecf96c9c0144bac946d1e90a8513794b56a8 binds Native bf047ce2e88e6c565eb6b80fae6559216524d41b in Source CI 37023611240. The workflow and every required job pass.

Retention now asks Native for canonical detached-activity closure before either run-detail pruning or external-payload reclamation deletes data. Pending/running work retains the original bytes. Changing only a mutable activity status to Completed does not release the hold. A canonical terminal event for the latest attempt does. An older installed Native preserves the newer history and payloads with activity_policy_backend_unsupported until compatible Native is restored.

Qualification Cases Assertions Errors / failures / skips
SQLite candidate source 83 5,343 0 / 0 / 0
MySQL candidate source 83 5,343 0 / 0 / 0
PostgreSQL candidate source 83 5,343 0 / 0 / 0
Affected protocol, quota and payload regressions 303 3,207 0 / 0 / 0

Each database includes the complete retention suite, 30 cases / 297 assertions. The new case checks pending and running detached operations, object-store bytes, the mutable-status negative case and canonical release. Local locked-package fallback also passes 1 / 38, zero errors, failures or skips. Affected Pint passes.

Separate locked-package PR CI 37023190620 passes the repository contracts and 2,376 cases / 48,461 assertions, zero errors or failures, 50 existing skips and six existing PHPUnit deprecations. Candidate Source does not replace that gate.

Raw JUnit, logs and exact source provenance are retained until 2026-12-31. Artifact SHA-256: 004838f4949e3603ed9bbbdb40a2595c72a8333eff6dea2f4b4d18c6259a1706.

JUnit SHA-256:

  • SQLite: 5559b5f6ec4b507753e445f6d2ca133d52d3ce1ca467e67a528f59ddb0191529
  • MySQL: e07f74d8285ba05125a59c5ebb337333b1fc42b2c46918634d5fffc6f31e4c43
  • PostgreSQL: fa12304f2b72efba92d6952f5df4c597cd69c26776e9cb89675aeba0120ea96d
  • Affected regressions: a6ac027bb68d8aacd9ee492937e528c8ae5bb0344a464f77e27c9a2f75728295

Commands and the read-only Native source overlay are in .github/workflows/phpunit-feature.yml at this Server commit. The primary command selects PreparedLocalActivityProtocolTest, HistoryRetentionTest and WorkerPollFenceTest with --fail-on-skipped. Hosted task containers are stopped.

This qualifies retention integration for the internal Source remote Activity lifetime. It does not admit public SDK Activity policies, prove physical callback exit or qualify local Abandon. This PR remains a draft and shared issue 136 stays open. Next work connects operation policy admission, complete local lifetimes and the required published mixed-language cascade with its coherent inspection view.

@rmcdaniel

Copy link
Copy Markdown
Member Author

Pending Activity cancellation reply qualified on Source

Exact Server 2fc521b5f132c5a6d0909ec6239f2fbbc435bae6, Native readonly source overlay bf047ce2e88e6c565eb6b80fae6559216524d41b.

Native can park cancellation delivery while an activity stop acknowledgment is outstanding. Server now returns HTTP 200 for that known pending response only when the workflow claim was explicitly released. Child pending responses retain the same rule. Other refused replies retain their error status. The worker OpenAPI is version 31 and documents both pending reasons. Published protocol 1.19 and its frozen fixtures remain unchanged.

The HTTP regression exercises child, remote Activity and prepared local Activity pending replies. It verifies the release, validates the schema and confirms that no canonical cancellation delivery event has been written yet. The preceding controller returns 409 for both Activity cases. The correction passes 3 cases / 195 assertions in the retained affected suite. This is route-contract proof, with the Native pending outcome injected at that boundary.

Exact Source CI passes every required job. Its isolated source selection passes 83 cases / 5,343 assertions on each of SQLite, MySQL and PostgreSQL, plus 305 affected cases / 3,337 assertions, with zero errors, failures or skips. The affected suite includes 31 cooperative protocol cases / 747 assertions and the OpenAPI contract checks. The retention regression remains covered on all supported databases.

Separate locked-package CI passes 2,378 cases / 48,587 assertions, zero errors or failures, with 50 existing skips and six existing PHPUnit deprecations. The locked-package lane does not claim the optional Native source bridge.

Commands are retained in the exact CI logs. The Source lane runs php vendor/bin/phpunit against the cooperative, workflow-worker, activity-worker, search validation, quota, payload transport/completion and OpenAPI regression files with --log-junit /evidence/affected-regression.xml --colors=never. Its separate source candidate selection runs on all three databases. Local affected Pint and 44 Source route/OpenAPI cases / 891 assertions pass.

Raw source evidence and provenance are retained until 2026-12-31. Artifact SHA256 ab4f56fe49ba4a9af445e648062a52ff6435f7f5554f3ec2add71a334e1df34d.

JUnit file SHA256
sqlite.xml bea13e8685897f013fb8c8a1bb55f03d2e079406787d2ee0de0e5d460a97f725
mysql.xml 55847b2dcc9d7f21b194dd26230fa0540596b91464bde734927df1201cc19438
pgsql.xml 4ba7cdfcf1a99acbd4579736885ef51fe538ef4b0f2cd7c37bd2b2854ba625ad
affected-regression.xml 1cd7b3cb689425545c039544ebb0e6ae59bf61da3edb4f5b2565d6442ca3e220

Hosted test containers were removed successfully. PHP, Python and Rust validators are qualifying the matching response on this tuple. The next product step is explicit portable Activity policy admission, then actual policy scenarios. This response correction does not itself add Activity policy authoring or prove the published mixed-language cascade. This PR remains draft and shared issue 136 remains open.

@rmcdaniel

Copy link
Copy Markdown
Member Author

Remote Activity policy admission qualified on Source

Exact Server 8940fb8a866e24d7874947e850f7fc864fbdfd3c with Native 9f8cb61947997c1c60f294bdf559d02d8258cc18.

Explicit remote Activity policies now require protocol 1.20, the original claim's cooperative capability and an installed bridge that implements their canonical semantics. Unsupported requests return activity_cancellation_policy_not_supported with the missing capabilities and remediation before scheduling. Registration changes cannot upgrade an existing claim. Remote Abandon requires a finite total timeout. Worker OpenAPI version 32 documents the three policies and historical defaults. Explicit local policy authoring remains refused pending its portable contract.

Exact Source CI passes every required job. The prepared-local, retention and fencing selection passes 83 cases / 5,343 assertions on each of SQLite, MySQL and PostgreSQL, with zero errors, failures or skips. The affected SQLite regression suite passes 311 / 3,376, including cooperative protocol checks 37 / 786. The new HTTP admission cases are in the affected SQLite suite. Native's seven new canonical admission cases separately pass on both MySQL and PostgreSQL.

Ordinary exact-head locked-package CI passes 2,384 cases / 48,637 assertions, with zero errors or failures, 50 existing skips and six existing deprecations. Its service cleanup passes.

Local PHP 8.4 Source route and OpenAPI checks pass 50 / 930, with zero errors, failures, warnings or skips. Separate locked-runtime fallback checks pass 6 / 46, verifying explicit unsupported-runtime diagnostics and no scheduling side effects. Affected Pint passes. Commands: vendor/bin/phpunit tests/Feature/Api/CooperativeCancellationProtocolTest.php tests/Feature/Api/WorkerOpenApiTest.php, the remote admission selection against locked dependencies, and vendor/bin/pint --test for the changed PHP files. Source tests bind the exact Native checkout through Composer's path-package overlay.

Raw Source JUnit, logs and provenance are retained until 2026-12-31. Artifact SHA256 f1d76581553e80d036c75ebfcee81d62f24d8646cfa803be73ca7dc3d1ab7527.

JUnit SHA256
Affected regressions 238774fabbaf6f36b1293acbf2f94b925c16a1f47eda2de5b7e1dbeda9cb1fe0
MySQL 7d9ba9b355f063d98a9c5b4f7ee74b16f85f3e5c355ac5a8a866976351c3d7a3
PostgreSQL 3d945dfcf54ccb10c2bae203aa49c63eb1bffec07272e5b7826387467c7225ee
SQLite f75818c4b4f97d8347859e2aa0c23577622f31b2680c317a3179d2a871a28a8c

Next: qualify SDK authoring, immutable policy replay and actual worker policy scenarios against this tuple. Explicit local policies, independent local Abandon lifetime, scopes, coherent cascade inspection and the exact published mixed-language scenario remain required. This PR stays draft and shared issue 136 remains open. Protocol 1.20 stays explicit and unfrozen. Published artifacts are unchanged.

@rmcdaniel

Copy link
Copy Markdown
Member Author

Next implementation: one inspectable cancellation cascade

This is the next implementation step after the mixed Source recovery case. It is planned, not implemented or qualified.

The existing Server run diagnostics omit the cancellation cascade. Native retains the canonical request, delivery, propagation and stop receipts. WorkflowCancelled currently records a human-readable reason for successful cleanup versus deadline expiry, without a separate structured cleanup outcome. Waterline's service detail already consumes Server run diagnostics. CLI's dw debug workflow consumes the same endpoint, but its human renderer needs a cancellation section.

Native and Server

  • Record structured cleanup completion or deadline expiry with the terminal cancellation event. Keep terminal workflow status separate from cleanup outcome. Termination and an attempt losing authority must remain distinguishable. A missing worker or lost lease cannot establish physical callback exit.
  • Add one canonical inspection serializer shared with embedded Waterline. Return original root metadata, each local request and run, propagation edges and admission/stop/recovery evidence. Resolve children through the existing indexed run/call relationships, preserving historical run selection. Do not substitute a current continued run for the originally targeted run.
  • Use bounded queries and explicit completeness/truncation indicators. Validate root identity and immutable budget against canonical events. Filter every resolved run by namespace before reading or returning it. Missing, pruned, conflicting or malformed evidence must be visible as incomplete rather than successful cleanup.
  • Present pending operations and policy outcomes, including Abandon branches that retain independent work. Distinguish an attempt being fenced from a matching canonical stop receipt and from an operation finishing before cancellation. Tie recovery to the original delivery boundary and attempts.

CLI and Waterline

Render that same contract in CLI human output and Waterline embedded/service detail. Show the original deadline, root and child identities, callback stop evidence, worker loss/replacement and cleanup outcome together. Keep JSON usable for automation. State safe recovery actions from the observed state, including whether work is still expected to recover or was explicitly abandoned.

Verification

Exercise real canonical completion and expiry, multiple levels, independent Abandon, conflicting roots, pruned/missing receipts, namespace isolation and query/response bounds. Verify SQLite, MySQL and PostgreSQL where the affected surfaces support them. Check CLI human/JSON and embedded/service UI parity.

Extend the mixed PHP/Python/Rust case to inspect the coherent API view during cleanup and after replacement, with the unchanged original 30-second budget. The final published case must verify the UI/API view as well as the runtime cascade. Issue 136 remains open until the complete published outcome and competitive evidence are demonstrated.

@rmcdaniel

Copy link
Copy Markdown
Member Author

Cancellation cascade diagnostics implemented, exact-head CI pending

Server 70807856b416b8792099108712cdd031e2c66c89 adds Native's shared cancellation cascade view to current-run and exact-run debug responses. It distinguishes runtime support from absence of a cooperative request, preserves historical run selection, and keeps namespace authorization ahead of related-run inspection. No mutation route or default protocol change is involved.

Native 88ea6e46499332427d0f9a439768ae0bf86e8702 is the exact source dependency. Its full Source qualification and PR qualification pass. Server's exact-head Source qualification and ordinary locked-package qualification are pending.

Local PHP 8.3.35 qualification in an ephemeral Docker container as UID/GID 1000:1000 passes:

  • Locked Native 2.3.3: WorkflowDebugTest, 12 / 165, no errors, failures or skips. Runtime support is false and the cascade is null.
  • Exact Native source overlay: new API cases plus WorkflowDebugTest, 18 / 287, no errors, failures or skips.
  • Complete Source selection: prepared locals, history retention, worker poll fences, new cascade API cases and existing debug cases, 101 / 5630, no errors, failures or skips.
  • Affected Pint and git whitespace checks pass. The tracked Composer lock is unchanged.

The six new API cases execute real request, claim, delivery and completion routes. They verify original budget and duplicate identity, child propagation from either selected run, completed cleanup versus deadline expiry, missing canonical request history, related namespace isolation and historical selection. Debug reads do not mutate canonical history. The latter two cases use deliberately inconsistent relationship/current-pointer fixtures to test read authorization and selection, not to claim execution behavior.

Exact command for the full local Source selection:

php vendor/bin/phpunit tests/Feature/PreparedLocalActivityProtocolTest.php tests/Feature/HistoryRetentionTest.php tests/Unit/WorkerPollFenceTest.php tests/Source/CancellationCascadeDiagnosticsTest.php tests/Feature/WorkflowDebugTest.php --fail-on-skipped --log-junit /evidence/server-source-all.xml --colors=never

Source CI executes that selection on SQLite, MySQL and PostgreSQL with the immutable Native commit input. Ordinary feature CI retains published Native 2.3.3. Local Source JUnit SHA-256 is 236dffeed1dfcc2d78fd0a7e7fd619335af1c807677adddf2c495b94d7ced9ff. Locked debug JUnit SHA-256 is 304891a4675cf0709e72bf73210fa79294715a2fc86f09d5bdc95325971cc2da.

Next: verify hosted raw evidence, render the same contract in CLI and embedded/service Waterline, and extend the mixed Source recovery case to inspect it during cleanup and replacement. All candidates remain drafts. Shared issue 136 remains open. Published artifacts and default worker protocol 1.19 are unchanged.

@rmcdaniel

Copy link
Copy Markdown
Member Author

Cancellation cascade diagnostic API qualified at the exact Server/Native tuple

Server 70807856b416b8792099108712cdd031e2c66c89 with Native 88ea6e46499332427d0f9a439768ae0bf86e8702 passes the three-database Source qualification and ordinary locked-package qualification. Both conclusions are success. Native's full Source and PR matrix also pass at that exact dependency. Candidates remain drafts and protocol 1.20 remains unfrozen.

Both current-run and exact-run debug endpoints now include the same canonical cancellation cascade view. Installed-runtime support is separate from absence of a request. Original root identity and deadline, local identities, child propagation, recorded delivery, cleanup outcomes, matching stop receipts and lease recovery are inspected together. Historical selection and namespace authorization are preserved. Diagnostics do not mutate cancellation state or infer callback exit from a fence.

Raw hosted results

Selection Tests Assertions Errors / failures / skips
Source, SQLite 101 5630 0 / 0 / 0
Source, MySQL 101 5630 0 / 0 / 0
Source, PostgreSQL 101 5630 0 / 0 / 0
New API cases on each database 6 122 0 / 0 / 0
Affected SQLite worker/payload/protocol regression 311 3376 0 / 0 / 0
Ordinary published Native 2.3.3 feature suite 2384 48635 0 / 0 / 50 existing

Ordinary feature CI also reports six existing PHPUnit deprecations. The new API cases cover original budget through duplicate and completed cleanup, child propagation viewed from either run, deadline expiry, missing canonical history, related namespace isolation and exact historical selection. Both deliberate inconsistent fixtures test read authorization and selection, not runtime execution.

The Source selection runs prepared locals, retention, poll fencing, cascade diagnostics and existing debug cases with --fail-on-skipped. SQLite took 16.484184 seconds, MySQL 37.645107 and PostgreSQL 39.364904. The new six API cases took 1.060965, 2.248536 and 2.914391 seconds respectively.

Raw JUnit, database logs and immutable source provenance are retained until December 31, 2026. Provenance matches both exact commits. Archive SHA-256 is 382b5a933e4574e5ac4d45d27af3ef1a2345d6c8989e021adf4ff83e8d9ba63b.

  • SQLite JUnit: 178f926de4595fbf3ece960612001c4edab0e6d3a3b44c47c8bbc43f495b9960.
  • MySQL JUnit: 8ec449b97f9bd6ceaa7d9a70e809dd5e34041170f546e4950d4c15de4360deeb.
  • PostgreSQL JUnit: 3b8bd9249ccfaad4138f18afce38f0dfea88ae9b798f598d170806807f3c27ab.
  • Affected regression JUnit: 967bf7d086ee66ca8c1e50f26a05caf4707f4af33986075806e8d0cf08ef7ae9.

Next customer-facing step

CLI human/JSON rendering is implemented and passes its local full suite. Exact-head CLI build qualification follows. Waterline embedded/service parity and the updated real mixed Source recovery case will inspect this API during cleanup, replacement and final closure. The previous mixed case names an older Native/Server tuple and does not qualify this API. Published/default protocol 1.19 and all published artifacts are unchanged. Shared issue 136 stays open until its complete published cascade and fair competitive evidence are demonstrated.

@rmcdaniel

Copy link
Copy Markdown
Member Author

Next cancellation diagnostics correction

The updated exact mixed Source qualification passed against Server 70807856b416b8792099108712cdd031e2c66c89. Its retained final API response confirms a diagnostic inconsistency:

  • The run is terminal Cancelled and both parent/child cleanup outcomes are completed within the original deadline.
  • execution.task_problem_badge correctly has code=history, label=Task History, tone=secondary, and says the run previously needed repair.
  • Generic findings still emits severity=warning, code=task_problem, and “The run summary has a task problem flag.”

WorkflowRunDiagnostics::findings() currently checks only the boolean and ignores the existing distinction between historical recovery and an active problem. Next: render the historical case as an informational recovery finding with a useful explanation, retain active repair/replay warnings and errors, and qualify those outcomes through the real debug API. Keep canonical failures and recovery evidence visible. The existing cascade view is accurate and remains unchanged by this display correction.

Raw artifact, file polyglot-evidence/dw-connected-cooperative-29a6d8afdc7be5b2/cascade-final.json, SHA-256 40aa9c1e63986c8ae98428d14e14cf7b9cb3c6fef64c038d13223f46b8ae78b0. This action is planned, not implemented yet. Server PR 291 stays draft and shared issue 136 stays open.

@rmcdaniel

Copy link
Copy Markdown
Member Author

Historical repair finding corrected, full qualification pending

Server 172dedb57a71629048fc145175d21eb626bf386d now uses the existing task-problem badge classification in generic debug findings:

  • Historical recovery becomes severity=info, code=task_recovery_history, with “The run previously needed workflow-task repair or replay recovery.”
  • Active or unclassified task problems retain the existing warning. Active replay failure retains its separate error.
  • Task/failure records, the history badge, terminal status and the cancellation cascade view remain visible.

Real HTTP debug API cases reproduce the defect on completed and cancelled runs: 3 cases / 29 assertions, two failures against preceding Server 70807856b416b8792099108712cdd031e2c66c89. The active replay case already passes before the repair.

The complete affected debug API file now passes 15 cases / 200 assertions, with zero errors, failures or skips, against locked published Native 2.3.3. Scoped Pint and whitespace checks pass. The tracked Composer lock is unchanged.

Local evidence SHA-256
Before log 32a55970258a70608e4fae2911b742162cf7a869d60fa48e39b499f39c6c6b36
Confirming API/style log a0130e35147e5237b67cc8f18bb5fd7b5a4c2b8d7a7b925c36c34507ea5022ba

Source 37074355176 binds Native 47d3ade12f8f51bbd75cbddf4aa714040f1a4195 and will run the actual API and affected prepared-local/cascade scenarios on SQLite, MySQL and PostgreSQL. Ordinary locked-package CI 37074348861 and the existing query/performance/public-boundary gates are pending. No full current-head hosted pass is claimed yet.

The complete mixed-language Source pass retains preceding Server 708078 and the exact current Native/PHP/Python/Rust/CLI tuple recorded there. It proves cleanup SIGKILL recovery under the original 30 seconds, while exposing the historical warning fixed here.

Next: inspect exact current Source/ordinary gates and retain raw results, remove task dependencies and scratch, then settle remaining model decisions and competitive/published qualification. PR 291 remains draft, shared issue 136 remains open, candidate protocol 1.20 remains unfrozen and published/default 1.19 is unchanged.

@rmcdaniel

Copy link
Copy Markdown
Member Author

Historical repair finding corrected and qualified

Server 172dedb57a71629048fc145175d21eb626bf386d now uses the existing task-problem badge classification in generic debug findings:

  • Historical recovery becomes severity=info, code=task_recovery_history, with “The run previously needed workflow-task repair or replay recovery.”
  • Active or unclassified task problems retain the existing warning. Active replay failure retains its separate error.
  • Task/failure records, the history badge, terminal status and the cancellation cascade view remain visible.

Real HTTP debug API cases reproduce the defect on completed and cancelled runs: 3 cases / 29 assertions, two failures against preceding Server 70807856b416b8792099108712cdd031e2c66c89. The active replay case already passes before the repair.

The complete affected debug API file now passes 15 cases / 200 assertions, with zero errors, failures or skips, against locked published Native 2.3.3. Scoped Pint and whitespace checks pass. The tracked Composer lock is unchanged.

Local evidence SHA-256
Before log 32a55970258a70608e4fae2911b742162cf7a869d60fa48e39b499f39c6c6b36
Confirming API/style log a0130e35147e5237b67cc8f18bb5fd7b5a4c2b8d7a7b925c36c34507ea5022ba

Source 37074355176 binds Native 47d3ade12f8f51bbd75cbddf4aa714040f1a4195 and passes 104 cases / 5665 assertions on each of SQLite, MySQL and PostgreSQL, with zero errors, failures or skips. Each database passes the three new diagnostic cases with 35 assertions. Affected SQLite regressions pass 311 / 3376, without errors, failures or skips.

Ordinary locked-package CI 37074348861 passes 2387 / 48674, with 50 existing skips, six existing deprecations and zero errors or failures. Repository contracts, concurrent query HTTP, bounded-growth and polling-cache smoke, and public-boundary checks pass. All required current-head hosted gates are green.

Raw three-database Source evidence and perf smoke artifacts are retained until December 31, 2026.

Retained Source artifact SHA-256
Source ZIP f93a1cf74e6f84961e6dfb1d3de4f6915f7b1d537daf9ae9d99b1b7c8a301289
Provenance 87ead6b3ccd2c131e10490504a35801edffba0515cc3d0ec9196933c0e044b21
SQLite JUnit b7a14315b2755850ac47f833cf33f5a292eff7817deec9ca444a0a33eaa27383
MySQL JUnit 0e281f08ed9d4a42559d9dad7240d349d0ff988ff22c1643413bc0a05b110fd8
PostgreSQL JUnit cad0d1dfb555e2baf468b3f6697da2570b4905a39f703c7cfd967a1726871366
Affected JUnit 78675532fb46cbad7841ab49a46ae6e7684335081da3b695724b135f4f48bcd5

The complete mixed-language Source pass retains preceding Server 708078 and the exact current Native/PHP/Python/Rust/CLI tuple recorded there. It proves cleanup SIGKILL recovery under the original 30 seconds, while exposing the historical warning fixed here.

Next: settle remaining model decisions and competitive/published qualification. Dependencies have been removed, and disposable qualification scratch will be removed after record verification. PR 291 remains draft, shared issue 136 remains open, candidate protocol 1.20 remains unfrozen and published/default 1.19 is unchanged. This Server component pass does not replace the exact published mixed-language/UI closure gate.

@rmcdaniel

Copy link
Copy Markdown
Member Author

Next Server local-policy admission and replacement compatibility

Native ba1aa4770c4b94533548bd45bd3ba028229c54e8 provides explicit prepared-local TryCancel / WaitCancellationCompleted admission and an installed-bridge policy list. Its local 90 / 725 and exact PR checks pass. Full Source 37077131786 is still running, with no failure conclusion claimed as qualified. Keep this Server PR a draft and use that exact Native source for the next consumer qualification.

Implement the following on the existing Server branch:

  1. Discover supported local policies from the actual bound optional prepared-local bridge. An older backend or alias must advertise no explicit local-policy support. Default protocol 1.19 remains unsupported.
  2. Give policy-aware workers a distinct prepared_local_activity_cancellation_policies capability, requiring existing cooperation/prepared-local capabilities and protocol 1.20. Bind it to the immutable original Server-issued workflow claim. A later registration edit cannot upgrade an already executing claim.
  3. Reject unsupported policy admission before callback preparation, payload resolution or partial group writes. Return the requested policy, installed policy list, worker identity, missing capability/runtime reason and useful remediation. Local Abandon remains explicitly refused.
  4. Filter replacement claims using canonical local Scheduled policy snapshots, both before probing and under the existing run lock. A worker lacking this capability cannot replay an explicitly authored local policy. Reuse the existing claim/replay boundaries and avoid a second work tracker or mutable inferred authority.
  5. Apply the same checks to preparation, recovery and complete local all-group admission. Preserve response-loss identity, original owner/attempt, root deadline and group atomicity.

Qualification must run real API cases against the exact Native source on SQLite, MySQL and PostgreSQL. Include positive TryCancel/Wait admission, older backend discovery/refusal, old claim plus re-registration, incompatible replacement and unchanged publication authority/history after refusal. Keep ordinary locked-package API tests green. Then add PHP/Python/Rust authoring/replay and connected supervisor checks before publishing this capability.

Existing Server 172dedb diagnostics qualification remains bound to its recorded Native 47d3ade and does not prove this new consumer work. Shared issue 136 remains open, protocol 1.20 stays unfrozen, and the exact published mixed-language/UI plus fair competitor gates still apply.

@rmcdaniel

Copy link
Copy Markdown
Member Author

Prepared local cancellation policies bound to issued worker authority

Server dd8996fdd6d488b48d575b1b934821bd8bedc8ed implements the recorded admission plan against qualified Native ba1aa4770c4b94533548bd45bd3ba028229c54e8.

Discovery reads the actual installed bridge's optional policy list. Older optional-role bridges and default protocol 1.19 advertise an empty list. The new worker capability prepared_local_activity_cancellation_policies requires prepared-local support, cooperation and protocol 1.20. Preparation/recovery checks the original immutable claim before payload resolution. Re-registration cannot upgrade an existing claim. Unsupported policies identify the requesting/original worker, requested policy, supported list, missing capability and remediation.

Atomic group admission checks every explicit local policy before any sibling is created and rechecks within the serialized mutation. Replacement claims check canonical local Scheduled policy before probing and under the existing run lock. An incompatible worker cannot replay that policy. Expired-lease maintenance can still record its repair request without admitting work to that worker. Omitted policy preserves historical admission and legacy replacement. Local Abandon and null are refused.

Local PHP 8.4.26 / locked Laravel 13.34.0, isolated SQLite results:

Selection Tests Assertions Errors/failures/skips
New policy HTTP cases 18 240 0
Complete Source lane 122 5,905 0
Affected regressions 311 3,376 0
OpenAPI/workflow/evolution contracts 27 264 0

The new cases include a backend that supports only one policy, response-loss retries, changed-policy refusal, old claim versus re-registration, refusal before malformed payload resolution, no partial group writes, and actual API replacement/recovery with unchanged total lifetime. They validate durable admission and claim authority, not physical SDK callback stops. Confirming scoped Pint and whitespace checks pass. Tracked Composer lock remains SHA-256 04aebc742ca108b4e2922b5f3301759f305304e852c57dc1e196fe89fd9b2497.

Local log SHA-256
New policies 5832efb8421a682d1248d4f0704447726da2aea8e5802d05e91fd0f6ba463c69
Source lane d3d4f8001977e1a671fb8ca0d2b1228f38ebffd22f0de5d50bb99a694bdab870
Affected regressions 88570bf5869584f44d014c9e2e76c20df8d8f457c207911b0047729118e642f4
Contracts 1264e6c29674b47173ccd125753cc50304eaa4d096e5472d2daded092f068eaf
Confirming Pint d4c0f98c6aabce5c15c404a55f0b15a9d62b82376009cd78e88ae22f2d743de2

Three-database Source 37079106466, ordinary locked-package CI 37079089323, query HTTP and performance are running. Hosted qualification is pending. The ordinary lane retains published Native 2.3.3. OpenAPI document revision 33 describes this unfrozen candidate without changing default/published protocol 1.19.

Next: inspect and retain exact hosted outcomes, then qualify PHP/Python/Rust authoring, replay and physical supervisors for these policies. The earlier successful mixed Source cascade retains its older exact tuple. Shared issue 136 stays open and the PR stays draft. Published artifacts/UI, the required original-budget mixed-language crash recovery outcome and demonstrated fair competitive advantage remain closure gates.

@rmcdaniel

Copy link
Copy Markdown
Member Author

Prepared-local policy admission passes exact hosted qualification

Server dd8996fdd6d488b48d575b1b934821bd8bedc8ed with Native ba1aa4770c4b94533548bd45bd3ba028229c54e8 passes three-database Source 37079106466. SQLite, MySQL and PostgreSQL each pass 122 tests / 5,905 assertions, with zero errors, failures or skips. All 18 new policy HTTP cases pass 240 assertions on each database. Affected SQLite regressions pass 311 / 3,376, without skips.

Ordinary locked-package CI 37079089323 retains published Native 2.3.3 and passes 2,387 / 48,736, with 50 existing skips, six existing deprecations and zero errors/failures. Repository/central contracts, query HTTP, performance bounded-growth/smoke and public boundaries pass at this head.

The tests qualify installed-backend policy discovery, original-claim capability admission, refusal before payload resolution, atomic group checks, canonical history compatibility, legacy omission, response-loss retries and replacement recovery without extending total lifetime. A restricted backend cannot replay a policy it does not implement. Local Abandon/null are refused. The claim filter preserves legitimate expired-lease maintenance. SDK authoring and physical supervision require their separate connected qualification.

Raw Source artifact 11257738182, retained until 2026-12-31, includes all three logs/JUnit files, affected-regression JUnit and exact source provenance.

Raw evidence SHA-256
Original ZIP 95eb16cab7fb772d4232a43478916a5c2af972fd6e38371f4292d8958d03c001
Provenance 8af9ac276ac0368148598bfce14ebf007cc115b16d37e27b5c67cbf7af3dddb5
SQLite JUnit a337bfea328db71de13ef3abac44a71d4890a77a0d24628c5146454cfcda9663
MySQL JUnit f93ce2e6fc16c940cb26b0d244aa9b05a9e0e4ddeb394219c46a73c95891b625
PostgreSQL JUnit 8eace9b31b84347301f8e31bfcd2031b5275a4ac72a2a471205f8d39a73a78d1
Affected regression JUnit 0b6854f23ee918c2c1ec96364db238ba3855db9303db6bc5e92a130ce2da885f

Next: PHP's policy authoring/replay/supervisor consumer, then Python/Rust consumers and the exact mixed original-budget recovery/UI qualification. The earlier successful mixed Source case retains its preceding exact tuple. Shared issue 136 remains open, this PR remains draft, protocol 1.20 is unfrozen and published/default 1.19 is unchanged. Exact published artifacts and a fair demonstrated competitive advantage remain required.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants