Skip to content

Python: fix: streamed chat completions with n>1 produce separate messages (Fixes #9779) - #9805

Closed
Dhruv Shaswat (Dhruv-Sh-git) wants to merge 1 commit into
microsoft:mainfrom
Dhruv-Sh-git:fix/issue-9779
Closed

Dhruv Shaswat (Dhruv-Sh-git) wants to merge 1 commit into
microsoft:mainfrom
Dhruv-Sh-git:fix/issue-9779

Conversation

@Dhruv-Sh-git

@Dhruv-Sh-git Dhruv Shaswat (Dhruv-Sh-git) commented Oct 28, 2026 •

Copy link
Copy Markdown

Fixes #9779. "fix: streamed chat completions with n>1 produce separate messages" --body "

What problem does this PR solve?

When a Chat Completions request uses n > 1, the OpenAI streaming client merged deltas from all choices into a single ChatResponseUpdate. This caused the final response to contain interleaved text instead of separate messages.
For example:

  • Expected: [A1A2, B1B2]
  • Actual: [A1B1A2B2]
    The non-streaming path already returns a separate message for each choice.

Root cause

_parse_response_update_from_openai() aggregated contents from all chunk.choices using contents.extend(choice_contents). This lost the separation between parallel completions before ChatResponse.from_updates could assemble the final messages.

Changes made

  • Updated the streaming loop in _chat_completion_client.py to emit one ChatResponseUpdate per choice, using choice.index to keep each completion independent.
  • Added a usage-only ChatResponseUpdate when chunk.usage is present.
  • Preserved the tool-call identity mapping for each choice.
  • Left the existing parsing helpers unchanged.
  • Added a regression test in test_chat_stream_merge_fix.py that verifies two streamed choices produce separate final messages.

Why this fixes the issue

Each choice now produces its own ChatResponseUpdate with a distinct message ID. ChatResponse.from_updates can therefore group updates correctly and build one assistant message per choice, matching the non-streaming behavior.

Testing

  • Added regression coverage for multiple streamed choices.
  • Existing tests passed locally with uv run pytest -q (0 failures).

Additional notes

  • No public API changes.
  • No new runtime dependencies.
  • The .NET implementation is unchanged."

Copilot AI balanced review requested due to automatic review settings October 28, 2026 17:00

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota.

@agent-framework-automation agent-framework-automation Bot added the python Usage: [Issues, PRs], Target: Python label Oct 28, 2026
@github-actions github-actions Bot changed the title fix: streamed chat completions with n>1 produce separate messages (Fixes #9779) Python: fix: streamed chat completions with n>1 produce separate messages (Fixes #9779) Oct 28, 2026
@Dhruv-Sh-git
Dhruv Shaswat (Dhruv-Sh-git) marked this pull request as ready for review October 28, 2026 17:02
@eavanvalkenburg

Copy link
Copy Markdown
Member

We do not support N > 1, closing this.

This branch was successfully deployed

1 active deployment
github-app-auth — eb085848 Deployed Oct 28, 2026 by Dhruv-Sh-git via team_check #7921
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

python Usage: [Issues, PRs], Target: Python

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Python: Streamed chat completions with n>1 merge all choices' texts into one message

3 participants