Repository navigation
Python: Streamed chat completions with n>1 merge all choices' texts into one message #9779
Description
Description
When a Chat Completions request is made with n > 1, the provider interleaves the choices' deltas in a single stream. The streaming parser merges every choice of a chunk into one update tagged with the shared chunk id (message_id=chunk.id), so the final ChatResponse concatenates the alternatives' texts in arrival order into a single message, while the non-streaming parser emits one assistant Message per choice.
Repro
from agent_framework import ChatResponse
from agent_framework_openai import OpenAIChatCompletionClient
from openai.types.chat.chat_completion_chunk import ChatCompletionChunk
client = OpenAIChatCompletionClient(model="test-model", api_key="test-key")
common = {"object": "chat.completion.chunk", "created": 1234567890, "model": "test-model"}
chunks = [
ChatCompletionChunk.model_validate({**common, "id": "multi", "choices": [
{"index": 0, "delta": {"role": "assistant", "content": "A1"}, "finish_reason": None},
{"index": 1, "delta": {"role": "assistant", "content": "B1"}, "finish_reason": None},
]}),
ChatCompletionChunk.model_validate({**common, "id": "multi", "choices": [
{"index": 0, "delta": {"content": "A2"}, "finish_reason": None},
{"index": 1, "delta": {"content": "B2"}, "finish_reason": "stop"},
]}),
]
updates = [client._parse_response_update_from_openai(c) for c in chunks]
print([m.text for m in ChatResponse.from_updates(updates).messages])
# actual: ['A1B1A2B2'] (one interleaved message)
# expected: ['A1A2', 'B1B2'] (per-choice messages, matching the non-streaming path)The non-streaming path (_parse_response_from_openai) already produces one Message per choice, so the streamed and non-streamed results disagree for the same request.
Tool calls are already choice-aware in streaming (tool_call_choice_index scoping), so this looks like an oversight for text rather than an intentional merge. Related: #9768 added per-message identity to the Responses client for the same class of aggregation problem.
Environment
- Package:
agent-framework-openai(chat completions path,OpenAIChatCompletionClient/RawOpenAIChatCompletionClient) - Observed on current
main(commit 6afcecd)
Activity
[-]Streamed chat completions with n>1 merge all choices' texts into one message[/-][+]Python: Streamed chat completions with n>1 merge all choices' texts into one message[/+]eavanvalkenburg commentedon Oct 28, 2026
We do not support N>1, closing.