Skip to content

Python: fix(core): keep AgentResponse usage when streaming a workflow agent - #9720

Open
Somay kousis (Somay-kousis) wants to merge 2 commits into
microsoft:mainfrom
Somay-kousis:fix/workflow-agent-stream-usage
Open

Somay kousis (Somay-kousis) wants to merge 2 commits into
microsoft:mainfrom
Somay-kousis:fix/workflow-agent-stream-usage

Conversation

@Somay-kousis

Copy link
Copy Markdown
Contributor

Motivation & Context

A workflow executor that yields an AgentResponse with usage_details (for example response = await agent.run(...) then await ctx.yield_output(response)) reports that usage through workflow.as_agent().run(...), but run(..., stream=True) ends with usage_details=None and no update carries a usage content. Streaming consumers of a workflow agent (hosting, DevUI, telemetry, cost tracking) lose the token counts. Details and repro in #9564.

Description & Review Guide

  • What are the major changes? In WorkflowAgent's conversion of an AgentResponse output into stream updates (_workflows/_agent.py), the usage is appended as Content.from_usage(...) to the last update, next to the other response-level fields copied there. That is how AgentResponse.to_updates() carries usage, and from_updates reads it back from contents. When none of the response's messages is shown to the caller (only internal function-call messages, or no messages), one update is emitted that carries the usage and the same response-level fields (continuation_token, finish_reason, agent_id, ...). It has no message_id or author_name, so it joins the message in progress instead of splitting it when it lands between chunks of another stream.
  • What is the impact of these changes? Streamed and non-streamed runs report the same usage_details. Usage from several AgentResponse outputs adds up the same way in both modes. Responses without usage are unchanged, and the usage-only update adds no message to the final response.
  • What do you want reviewers to focus on? The usage-only update for the no-caller-facing-message case: whether you'd rather attach that usage some other way.

Tests in tests/workflow/test_workflow_agent.py:

  • test_workflow_as_agent_stream_preserves_agent_response_usage: usage only on the last update, final response matches non-streaming.
  • test_workflow_as_agent_stream_sums_usage_of_multiple_agent_responses: two outputs, same total in both modes.
  • test_workflow_as_agent_stream_keeps_usage_without_caller_facing_messages[only_internal_call|no_messages]: usage, text and message count match non-streaming.
  • test_workflow_as_agent_stream_usage_only_update_does_not_split_a_message: a usage-only response between two chunks of one streamed message leaves it as one message.
  • test_workflow_as_agent_stream_usage_only_update_keeps_continuation_token: the usage-only update carries the response's continuation_token.

With _agent.py from main, 6 of these 7 fail. Removing only the usage-only branch fails the 4 tests that need it. tests/workflow: 1464 passed, 1 skipped, 2 xfailed; packages/orchestrations/tests: 297 passed, 6 skipped. Ruff check is clean on the changed files, and the core typing and test-typing tasks (pyrefly, ty, zuban, pyright, mypy) pass.

Related Issue

Fixes #9564

AI Assistance

  • No material AI assistance was used.
  • This is an AI-assisted contribution. I reviewed, understood, and verified all submitted content and accept responsibility for it.

AI assistance details: AI tools helped find the bug and write the code, the tests and this description. I reviewed all of it and ran the repro, the workflow tests, and the lint and typing checks myself.

Contribution Checklist

  • The code builds clean without any errors or warnings
  • All unit tests pass, and I have added new tests where possible
  • The PR follows the Contribution Guidelines
  • This PR links to an agreed issue with no competing open PR, or the Related Issue section documents a trivial-change or repository-automation exception.
  • This is not a breaking change. If it is a breaking change, add the breaking change label (or add "[BREAKING]" to the title prefix, before or after any language prefix) — a workflow keeps the label and title prefix in sync automatically.

… agent

WorkflowAgent turned an AgentResponse output into stream updates without
its usage, so a streamed run lost the token counts that a non-streamed
run reports. Carry the usage as usage content on the last update, the
way AgentResponse.to_updates() does. When none of the response's
messages is shown to the caller (only internal calls, or no messages),
emit one update for the usage and the response-level fields. It has no
message_id or author_name, so it joins the message in progress instead
of splitting it.

Fixes microsoft#9564

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Usage-only updates expose filtered internal messages through their raw representation.

1 open issue
What changed in this PR

This PR addresses #9564 by preserving token usage when workflow agents stream AgentResponse outputs, aligning streaming behavior with non-streaming runs.

Changes:

  • Appends usage content to the final update for each response.
  • Emits a usage-only update when no messages remain after filtering, preserving response metadata.
  • Adds regression tests for usage totals, filtered responses, message continuity, and continuation tokens.
File Description
python/​packages/​core/​tests/​workflow/​test_workflow_agent.py Adds streaming usage and metadata regression tests.
python/​packages/​core/​agent_framework/​_workflows/​_agent.py Preserves response usage in normal and usage-only stream updates.

Review effort: Balanced

response_id=data.response_id or response_id,
created_at=data.created_at
or datetime.now(tz=timezone.utc).strftime("%Y-%m-%dT%H:%M:%S.%fZ"),
raw_representation=cast(AgentResponse[Any], data),

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, fixed in 56d3841: the usage-only update no longer sets raw_representation, so the filtered messages don't reach the caller through it or the final response. The edge-case test now also asserts that no streamed update has a raw representation and that the final response's raw representation holds no AgentResponse; it fails on aa19c4f and passes now.

…pdate

The usage-only update carried the whole AgentResponse as its raw
representation, so the internal messages that the caller filter drops
reached the caller through it and through the final response. Leave it
unset, and assert that no update or final raw representation carries
the response.

This branch was previously deployed

1 inactive deployment
github-app-auth — 56d3841f Deployed Oct 26, 2026 by Somay-kousis via add_label #26201
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

python Usage: [Issues, PRs], Target: Python

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Python: [Bug]: WorkflowAgent drops usage_details of an AgentResponse output when run with stream=True

2 participants