diff --git a/docs/specs/004-python-function-calling-loop.md b/docs/specs/004-python-function-calling-loop.md index 8381d6b65c6..8311cc45fb7 100644 --- a/docs/specs/004-python-function-calling-loop.md +++ b/docs/specs/004-python-function-calling-loop.md @@ -434,6 +434,10 @@ that manually replay messages own the equivalent rule: do not resend an approval original shape and fails explicitly when that provider representation is unavailable or inconsistent. Locally generated shell outputs preserve stdout, stderr, truncation notices, non-zero exits, and timeout outcomes and are sent in both service-side and stateless continuation modes. + A `shell_call` for a configured local executor with valid item/call identifiers and `commands=[]` is a no-op: + return the call paired with a `shell_call_output` containing `output=[]`, without entering local execution or + approval. Preserve `max_output_length` when supplied. This applies to explicit local environments and omitted + environments resolved to the configured local executor; hosted, unconfigured, and malformed calls are unchanged. ### Reasoning-bound calls @@ -592,6 +596,7 @@ that manually replay messages own the equivalent rule: do not resend an approval | Parallel calls | Results retain the corresponding call ids and execution count. | `test_max_function_calls_limits_parallel_invocations`, `test_streaming_multiple_function_calls_parallel_execution` | | Sequential calls | Calls execute one at a time in model order, including after approval replay; mixed local/Host-owned siblings retain that order, approval-only batches wait for every decision before executing, each call has isolated context, and middleware termination takes effect after the accepted batch finishes. | `packages/core/tests/core/test_tools.py::test_sequential_function_invocation_runs_calls_in_model_order`, `test_sequential_function_invocation_isolates_context`, `test_sequential_function_invocation_finishes_batch_after_termination`, `packages/core/tests/core/test_harness_tool_approval.py::test_mixed_batch_hides_already_approved_request_until_approval_replay`, `test_sequential_approval_replay_preserves_model_order_when_responses_are_reversed`, `test_sequential_approval_replay_waits_for_the_complete_batch`, `packages/core/tests/core/test_function_invocation_logic.py::test_sequential_mixed_batch_preserves_approval_and_host_model_order`, `packages/core/tests/workflow/test_agent_executor_tool_calls.py::test_workflow_cancels_host_member_of_mixed_batch_with_terminal_result`, `test_workflow_host_cancellation_preserves_occurrence_with_reused_call_id` | | Informational-only call | The call is returned but not executed or approved. | `test_informational_only_function_call_is_not_invoked`, `test_informational_only_function_call_does_not_request_approval`, `test_streaming_informational_only_function_call_is_not_invoked` | +| Empty local shell call | An empty command list for a configured local executor returns a paired empty shell output without execution or approval; both parsers preserve the pair and stateless replay preserves output limits. Hosted, unconfigured, and malformed calls do not receive synthetic results. | `packages/openai/tests/openai/test_openai_chat_client.py::test_empty_local_shell_call_replays_with_empty_output`, `test_empty_local_shell_call_parsing_emits_pair_without_execution`, `test_empty_shell_result_is_not_synthesized_for_hosted_unconfigured_or_malformed_calls` | | OpenAI hosted/local shell boundary | Hosted shell calls remain informational in streaming and non-streaming responses even when a local executor is configured; only valid explicit local-shell items or shell calls marked with a local environment can execute, local execution preserves its configured approval mode, stateless loops preserve the complete provider shell transcript or fail explicitly, and locally generated shell outputs preserve stdout, stderr, truncation notices, non-zero exits, and timeout outcomes in every continuation mode. | `packages/openai/tests/openai/test_openai_chat_client.py::test_response_content_creation_with_shell_call_remains_hosted_with_local_tool`, `test_parse_chunk_from_openai_shell_call_done_remains_hosted`, `test_parse_chunk_from_openai_local_environment_shell_call_done_emits_command`, `test_mixed_shell_calls_only_invoke_explicit_local_shell_call`, `test_stateless_shell_transcript_without_provider_item_fails`, `test_prepare_messages_keeps_local_shell_output_under_storage`, `test_shell_output_payloads_use_structured_function_result_items`, `test_malformed_local_environment_shell_call_is_not_executable`, `test_response_content_creation_with_local_shell_call_maps_to_function_call`, `test_malformed_local_shell_call_is_not_executable`, `test_response_function_call_named_local_shell_is_informational`, `test_parse_chunk_function_call_named_local_shell_is_informational`, `test_local_shell_tool_requires_approval_before_function_loop_execution` | | Declaration-only call | The call is surfaced as user input and is not executed; streaming arguments appear once while finalized request metadata remains available. | `test_declaration_only_tool`, `test_streaming_declaration_only_tool_preserves_metadata_without_duplicate_arguments` | | Declaration-only mixed batch | Known local siblings are never converted into Host pauses. With an authoritative session they remain deferred until every Host pause occurrence is answered, survive session serialization with a process-portable logical-run budget (including a reclassified approval continuation), reclassify against the current tool policy, execute exactly once using the configured concurrent or sequential invocation mode, replay results in original model order, and remain inert on replay; without one they execute on the same turn under the same configured invocation mode. Streaming updates and finalization preserve one Host pause and one result per local sibling. | `test_mixed_declaration_only_batch_defers_siblings_until_all_host_results`, `test_mixed_declaration_only_deferred_call_keeps_ordinary_middleware_semantics`, `test_mixed_declaration_only_deferred_call_reclassifies_changed_approval_policy`, `test_mixed_declaration_only_budget_survives_reclassification_to_approval`, `test_mixed_declaration_only_deferred_call_does_not_execute_removed_tool`, `test_mixed_declaration_only_budget_is_process_portable_without_approval_middleware`, `test_sessionless_mixed_declaration_only_batch_executes_siblings_sequentially`, `test_sessionless_mixed_declaration_only_batch_executes_siblings_concurrently`, `test_streaming_sessionless_mixed_declaration_only_batch_returns_pause_and_result` | diff --git a/python/packages/openai/agent_framework_openai/_chat_client.py b/python/packages/openai/agent_framework_openai/_chat_client.py index ea9b3e010ef..04a94b96906 100644 --- a/python/packages/openai/agent_framework_openai/_chat_client.py +++ b/python/packages/openai/agent_framework_openai/_chat_client.py @@ -2596,6 +2596,21 @@ def _shell_item_to_contents(self, item: Any, local_shell_tool_name: str | None) raw_representation=item, ) ) + if ( + local_shell_tool_name + and is_local_environment + and shell_call_id + and shell_call_item_id + and isinstance(raw_shell_commands, list) + and not raw_shell_commands + ): + contents.append( + Content.from_shell_tool_result( + call_id=shell_call_id, + outputs=[], + max_output_length=shell_max_output, + ) + ) elif item_type == "local_shell_call": raw_local_call_id = getattr(item, "call_id", None) local_call_id = raw_local_call_id if isinstance(raw_local_call_id, str) else "" @@ -2853,6 +2868,16 @@ def _prepare_shell_transcript_item_for_openai( expected_type: Literal["shell_call", "shell_call_output", "local_shell_call"], ) -> dict[str, Any]: """Restore a provider-issued shell transcript item for stateless replay.""" + if ( + expected_type == "shell_call_output" + and content.raw_representation is None + and content.call_id + and content.outputs == [] + ): + output: dict[str, Any] = {"type": "shell_call_output", "call_id": content.call_id, "output": []} + if content.max_output_length is not None: + output["max_output_length"] = content.max_output_length + return output payload = cls._serialize_provider_payload(content.raw_representation) if isinstance(payload, Mapping): typed_payload = cast("Mapping[str, Any]", payload) diff --git a/python/packages/openai/tests/openai/test_openai_chat_client.py b/python/packages/openai/tests/openai/test_openai_chat_client.py index f0da16d8a60..9a064bfe813 100644 --- a/python/packages/openai/tests/openai/test_openai_chat_client.py +++ b/python/packages/openai/tests/openai/test_openai_chat_client.py @@ -2589,6 +2589,127 @@ def test_malformed_local_environment_shell_call_is_not_executable( assert all(content.type != "function_call" for content in contents) +@pytest.mark.parametrize("max_output_length", [None, 4096]) +@pytest.mark.parametrize("round_trip", [False, True]) +@pytest.mark.parametrize("environment", [{"type": "local"}, None]) +def test_empty_local_shell_call_replays_with_empty_output( + max_output_length: int | None, round_trip: bool, environment: dict[str, str] | None +) -> None: + """Zero local commands must replay with a matching empty result, not an orphan call.""" + client = OpenAIChatClient(model="test-model", api_key="test-key") + item = ResponseFunctionShellToolCall.model_validate({ + "type": "shell_call", + "id": "sh_empty", + "call_id": "call_empty", + "status": "completed", + "action": {"commands": [], "max_output_length": max_output_length}, + "environment": environment, + }) + + contents = client._shell_item_to_contents(item, "run_shell") + + assert all(content.type != "function_call" for content in contents) + message = Message("assistant", contents) + if round_trip: + restored_result = Message.from_json(Message("assistant", [contents[1]]).to_json()).contents[0] + message = Message("assistant", [contents[0], restored_result]) + replay = client._prepare_messages_for_openai( + [message, Message("user", "Continue")], + request_uses_service_side_storage=False, + ) + + shell_calls = [entry for entry in replay if entry["type"] == "shell_call"] + assert len(shell_calls) == 1 + assert shell_calls[0]["call_id"] == "call_empty" + assert shell_calls[0]["action"]["commands"] == [] + shell_outputs = [entry for entry in replay if entry["type"] == "shell_call_output"] + expected_output: dict[str, Any] = {"type": "shell_call_output", "call_id": "call_empty", "output": []} + if max_output_length is not None: + expected_output["max_output_length"] = max_output_length + assert shell_outputs == [expected_output] + + +@pytest.mark.parametrize("streaming", [False, True]) +def test_empty_local_shell_call_parsing_emits_pair_without_execution(streaming: bool) -> None: + client = OpenAIChatClient(model="test-model", api_key="test-key") + executed_commands: list[str] = [] + + def local_exec(command: str) -> str: + executed_commands.append(command) + return command + + local_tool = OpenAIChatClient.get_shell_tool(func=local_exec) + item = ResponseFunctionShellToolCall.model_validate({ + "type": "shell_call", + "id": "sh_empty", + "call_id": "call_empty", + "status": "completed", + "action": {"commands": []}, + "environment": {"type": "local"}, + }) + if streaming: + event = ResponseOutputItemDoneEvent( + type="response.output_item.done", item=item, output_index=0, sequence_number=1 + ) + update = client._parse_chunk_from_openai(event, options={"tools": [local_tool]}, function_call_ids={}) + finalized = ChatResponse.from_updates([update]) + assert [content.type for content in finalized.messages[0].contents] == ["shell_tool_call", "shell_tool_result"] + contents = update.contents + else: + response = MagicMock( + output=[item], + output_parsed=None, + metadata={}, + usage=None, + id="resp_empty", + model="test-model", + created_at=1000000000, + status="completed", + incomplete=None, + ) + parsed = client._parse_response_from_openai(response, options={"tools": [local_tool]}) + contents = parsed.messages[0].contents + + assert [content.type for content in contents] == ["shell_tool_call", "shell_tool_result"] + assert [content.call_id for content in contents] == ["call_empty", "call_empty"] + assert contents[1].outputs == [] + assert executed_commands == [] + + +@pytest.mark.parametrize( + ("environment", "tool_name", "item_id", "call_id", "commands"), + [ + ({"type": "container_reference"}, "run_shell", "sh_empty", "call_empty", []), + ({"type": "local"}, None, "sh_empty", "call_empty", []), + (None, None, "sh_empty", "call_empty", []), + ({"type": "local"}, "run_shell", None, "call_empty", []), + ({"type": "local"}, "run_shell", "sh_empty", None, []), + ({"type": "local"}, "run_shell", "sh_empty", "call_empty", None), + ({"type": "local"}, "run_shell", "sh_empty", "call_empty", [1]), + ], +) +def test_empty_shell_result_is_not_synthesized_for_hosted_unconfigured_or_malformed_calls( + environment: dict[str, str] | None, + tool_name: str | None, + item_id: str | None, + call_id: str | None, + commands: Any, +) -> None: + client = OpenAIChatClient(model="test-model", api_key="test-key") + item = SimpleNamespace( + type="shell_call", + id=item_id, + call_id=call_id, + environment=environment, + action=SimpleNamespace(commands=commands), + status="completed", + ) + + contents = client._shell_item_to_contents(item, tool_name) + + assert [content.type for content in contents] == ["shell_tool_call"] + + @pytest.mark.asyncio async def test_local_shell_tool_requires_approval_before_function_loop_execution() -> None: """An explicit local shell call executes only after approval."""