Fix MCP session teardown races behind two flaky cloud e2e tests - #2060
Draft
RhysSullivan wants to merge 2 commits into
Draft
RhysSullivan wants to merge 2 commits into
RhysSullivan wants to merge 2 commits into
Conversation
Contributor
Cloudflare preview
Sign-in is Cloudflare Access (one-time PIN to an allowed email). The preview has its own database and encryption key; it is destroyed when this PR closes. |
@executor-js/cli
@executor-js/config
@executor-js/execution
@executor-js/sdk
@executor-js/codemode-core
@executor-js/runtime-quickjs
@executor-js/plugin-file-secrets
@executor-js/plugin-graphql
@executor-js/plugin-keychain
@executor-js/plugin-mcp
@executor-js/plugin-onepassword
@executor-js/plugin-openapi
executor
commit: |
Deploying with
|
| Status | Name | Latest Commit | Preview URL | Updated (UTC) |
|---|---|---|---|---|
| ✅ Deployment successful! View logs |
executor-marketing | b5daec8 | Commit Preview URL Branch Preview URL |
Sep 18 2026, 07:51 PM |
Deploying with
|
| Status | Name | Latest Commit | Updated (UTC) |
|---|---|---|---|
| ✅ Deployment successful! View logs |
executor-cloud | b5daec8 | Sep 18 2026, 07:52 PM |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two cloud e2e tests failed intermittently on main with no related code change. Both trace to the session Durable Object's teardown and restore paths, and one to a test-timing assumption.
mcp-destroyed-session-envelope — in-flight requests answered with a Vite 500 (
no such table: cf_agents_mcp_servers). The agents SDK'sdestroy()drops its tables anddeleteAll()s storage (removing the destroy-pending marker) before it aborts the isolate from asetTimeout(0). An owner check landing in that gap saw no marker, still held the cached session meta, and tried to restore the runtime, which starts by reading the dropped table. The DO now recordsdestroyingbefore any await indestroy()and answersterminatedfrom it, same as the durable marker.mcp-session-cap-eviction — timed out at 180s. Two causes:
capsessions in waves, and after every wave touches every session opened so far with a JSON-RPCping, which re-arms the idle alarm through the owner check every request goes through; a session is never left untouched for longer than one open+touch tick (measured and printed next to the window). By the cap arithmetic, admissioncap + 1is the first that can evict one of these sessions whatever else is resident (the comment in the test derives it), so the remaining 10 are then opened untouched at full width and the first of them forces the eviction. Opens and touches never overlap and every wave is at mostMAX_CONCURRENT_BUILDSwide (now exported fromsession-build-semaphore.tsand imported by the test), as are the cleanup DELETEs (which restore a disposed runtime before destroying it), so nothing the scenario does ever waits in that queue. The elapsed-time check is a diagnostic line now, not an assertion.validateMcpSessionOwnerrestored the runtime by callingonStartdirectly, which leaves PartyServer's own "started" state unset; the followingfetchre-ranonStartunderblockConcurrencyWhile, tearing down and rebuilding the runtime with the input gate held. Every restore through an RPC on a fresh instance paid for two cold builds. RPC restores now go through__unsafe_ensureInitialized(the SDK's own gate for RPC entry points) on a fresh instance, and restore in place otherwise.Observed, not fixed here. The same CI runs showed 30s
blockConcurrencyWhileresets (32concurrency_resetper run). Span timings point at the build semaphore's hand-off rather than the double build: every init that queued atsession-build-semaphore.tssat out the full 10s queue timeout, and the ones granted a slot from another Durable Object's request context did not finish before the 30s reset. That is tracked in #2063; the scenario stays out of the queue so it cannot hit it.Verified locally: DO unit tests (51), host-cloudflare and e2e typechecks, oxfmt, oxlint, and both e2e files against a full cloud boot (vitest globalsetup,
MCP_SESSION_TIMEOUT_MS=3000, motel span store). The cap-eviction scenario, fresh boot each time:mcp-destroyed-session-envelopefirst, three boots: kept 24 sessions resident through 6 open+touch ticks in 1681 / 1829 / 2061 ms (longest tick 394 / 444 / 539 ms against the 3000 ms window, 84 touches each); the 10 crossing opens took 355 / 359 / 430 ms; both files passed every time.MCP_SESSION_TIMEOUT_MS=1000, so the touched phase is twice the window: 2035 ms, longest tick 419 ms; passed.