Conversation
Only MESSAGE_SEND was in RPC_SYNC_METHODS, so every other JSON-RPC method took
the background branch in base_acp_server._handle_jsonrpc: the handler was
dispatched with asyncio.create_task and the server immediately answered
{"status": "processing"}.
That loses work silently for the two methods that talk to Temporal. A caller
doing task/create followed by event/send gets its task id back before
TemporalACP.handle_task_create has run start_workflow, so the signal can arrive
before the workflow exists and is dropped with "workflow not found". The caller
sees success either way, because the response was sent before the handler ran
and _process_request only logs the exception.
Observed in an eval harness as workflows that start, never receive their
payload, and sit on their wait timer until the client's timeout fires.
Both handlers are short Temporal RPCs (start_workflow, send_signal) and the
caller applies its own timeout, so awaiting them is cheap. The sync path also
returns a real JSON-RPC error when the handler raises, which makes the failure
visible and retryable.
Two properties the fix depends on, exercised through SyncACP so the test needs no Temporal connection and no network: - the handler finishes before the response is sent, so a caller that sequences task/create then event/send gets the ordering it asked for - a handler that raises produces a JSON-RPC error rather than a success, so the failure is visible and retryable at the client Each async handler yields with `await asyncio.sleep(0)` first, so a regression back to background dispatch loses the ordering race and fails the test.
JSONRPCResponse.error is typed JSONRPCError | None, so pydantic revalidates the dict the server passes in and the attribute is a model. Indexing it raised TypeError: 'JSONRPCError' object is not subscriptable. _handle_jsonrpc has no return annotation, so pyright could not see result or error either. Route the calls through a helper that asserts the type, which narrows it for the type checker and removes the twelve reportAttributeAccessIssue errors.
CreateTaskParams declares `params: dict[str, Any] | None = Field(None, ...)` with a positional default, which pyright does not read as a default, so it treats the field as required. Pydantic disagrees and reports it optional. Pass it explicitly rather than change the model, which is outside this PR's scope.
Making event/send synchronous broke batching. The 080_batch_events tutorial sends events in quick succession and asserts the workflow drains more than one per batch; awaiting each send serialises them, so no batch ever held more than one event and two of its tests failed. Background dispatch there is deliberate design, not an oversight. task/create is the call that actually needed fixing. Its contract is to hand back an id for a task that now exists, and its handler is what starts the workflow, so answering before the handler ran returned an id the server could not yet route to. With it synchronous the workflow is addressable before any event is sent, which is all the race needed. Tests now pin both directions: task/create awaits its handler and surfaces a handler failure as a JSON-RPC error, and event/send still acknowledges without waiting. Not fixed here: a send to a genuinely dead target still fails silently, because the background path logs the handler's exception and there is no status method for a caller to ask. That needs a design which reports failures without serialising sends, and is left for separate work.
Comment on lines
+139
to
+143
| # EVENT_SEND deliberately stays asynchronous. Callers send events in quick | ||
| # succession and the workflow drains them as a batch; awaiting each send | ||
| # serialises them and no batch ever holds more than one event. Once TASK_CREATE | ||
| # is synchronous the workflow is addressable before any event is sent, which is | ||
| # what the race needed. |
There was a problem hiding this comment.
EVENT_SEND still returns {"status": "processing"} before send_signal runs. If Temporal rejects the signal, _process_request only logs the error. Await EVENT_SEND so the caller gets a JSON-RPC error and can retry. Separate HTTP requests can still run at the same time, so waiting does not prevent callers from sending events quickly.
Prompt To Fix With AI
This is a comment left during a code review.
Path: src/agentex/protocol/acp.py
Line: 139-143
Comment:
`EVENT_SEND` still returns `{"status": "processing"}` before `send_signal` runs. If Temporal rejects the signal, `_process_request` only logs the error. Await `EVENT_SEND` so the caller gets a JSON-RPC error and can retry. Separate HTTP requests can still run at the same time, so waiting does not prevent callers from sending events quickly.
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
task/createwas dispatched in the background, so the ACP server handed back a task id before its handler had started the Temporal workflow. A caller that followedtask/createwithevent/sendcould have its signal arrive first and be dropped withworkflow not found, while seeing success on both calls. This makestask/createsynchronous and leavesevent/sendalone.What changed
src/agentex/protocol/acp.py—RPC_SYNC_METHODSgainsTASK_CREATE.Only
MESSAGE_SENDwas in that list, so everything else took the background branch inBaseACPServer._handle_jsonrpc:task/createlanded there by default rather than by decision. Its contract is to return an id for a task that now exists, andTemporalACP.handle_task_createis what runsstart_workflow, so answering early returns an id the server cannot yet route to. Nothing batches task creations, so there is no benefit being traded away, andstart_workflowis a short RPC.tests/test_acp_sync_dispatch.py— new, and pins both directions so neither is changed by accident:task/createcompletes its handler before responding, and a raising handler yields a JSON-RPC error rather than a success.event/sendstill acknowledges with{"status": "processing"}before its handler runs.The
task/createhandlers yield withawait asyncio.sleep(0)first, so a regression to background dispatch loses the ordering race and fails.Why
event/sendstays asynchronousAn earlier revision of this PR made
event/sendsynchronous too, and it broke batching. The10_async/00_base/080_batch_eventstutorial sends events in quick succession and asserts the workflow drains more than one per batch:Awaiting each send serialises them, so no batch ever holds more than one event. Two of that tutorial's tests failed, reproducibly. Background dispatch there is deliberate design.
Once
task/createis synchronous the workflow is addressable before any event is sent, which is what the race actually needed. The two calls have different contracts:event/sendis a legitimate fire-and-forget,task/createis a creation call.Review guide
Start with
src/agentex/protocol/acp.py; the change is one line and a comment. The behaviour it selects is insrc/agentex/lib/sdk/fastacp/base/base_acp_server.pyaround_handle_jsonrpcand_process_request, unchanged here but worth reading alongside. The handlers are insrc/agentex/lib/sdk/fastacp/impl/temporal_acp.py.Compatibility
The synchronous path returns the handler's return value, and
handle_task_createreturnsNone. The agentex backend reads this asreturn rpc_response.result or {}and raises only onrpc_response.error, so aNoneresult is unchanged behaviour for it. Its HTTP call to the ACP already carries a 60 s timeout, which bounds the new blocking.Not fixed here
A send to a genuinely dead target — a completed or terminated workflow — still fails silently.
handle_event_sendraises, but the raise lands in_process_request, which only logs, andRPCMethodhas no status method for a caller to ask after the fact. Reporting that without serialising sends needs its own design, so it is left for separate work.How this was found
An eval harness lost work to it repeatedly: workflows started, never received their payload, and sat on their wait timer until the client's timeout fired. Sixteen occurrences in one day across two agents, each costing a lost item and the full client timeout.
Testing
I could not run the suite locally:
uvhere is 0.8.13 against the repo'srequired-version >=0.9, and this machine's pip index needs credentials I do not have. The new tests need neither Temporal nor network, so CI covers them.This PR is not safe to merge because
EVENT_SENDcan lose an event while telling the caller it succeeded.Fix with agent prompt
Summary
ACP task creation now waits for its Temporal workflow to start before replying, so a following event can reach an existing workflow. Event sending stays asynchronous so callers can continue sending events without waiting for each one.
Diagram
sequenceDiagram participant C as Caller participant A as ACP server participant T as Temporal C->>A: task/create A->>T: start_workflow T-->>A: workflow started A-->>C: JSON-RPC result C->>A: event/send A-->>C: "{status: processing}" A->>T: send_signal in background alt signal succeeds T-->>A: accepted else signal fails T-->>A: error A->>A: log error only endReviews (4) · Last reviewed commit: "fix(acp): keep event/send asynchronous, ..."