Skip to content

fix(openai): mark interrupted agents and truncated chat streams as errors - #536

Open
nik13 wants to merge 17 commits into
devfrom
fix/opus-followup-docs-1069
Open

nik13 wants to merge 17 commits into
devfrom
fix/opus-followup-docs-1069

Conversation

@nik13

@nik13 nik13 commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Follow up on the recall of future-agi/docs#1069 without duplicating existing OpenAI lifecycle work.

  • Handle KeyboardInterrupt, asyncio.CancelledError, and other non-Exception aborts inside FITracer.start_as_current_span, before OTel ends the span. Respect the exception/status/end flags, preserve context restoration and the original exception, and apply FiSpan's existing error privacy. Ordinary exceptions remain OTel-owned and are not recorded twice.
  • Track per-choice chat finish reasons. EOF, an empty stream, a usage-only stream, or context exit before every requested choice completes is ERROR, not OK. Preserve successful tool-call/length/filter terminal responses, multiple choices, usage tails, and all SDK chunks unchanged.
  • Reuse the accumulator get_status hook already introduced by fix(openai): protect Responses privacy and finalize interrupted calls #520. Do not duplicate its request cancellation or explicit stream-close implementations.

Existing PR dependency and scope

Includes #520 at 3e19fa3b through a normal merge. The shared _request.py conflict is reconciled: retain #520's lazy raw-response/privacy handling and this PR's effective-body chat completion count. Both PRs remain open; landing #520 first keeps review scope clear. This head is the tested combined tree, not an untested merge suggestion.

I also checked #411 (audio explicit closes) and #344 (generator decorators). Neither handles these remaining core manual-context and chat terminal-state gaps.

Tracing stays passive: a truncated SDK iterator still yields the same partial chunks; this patch does not turn telemetry into an application exception. The application completion guard is now fixed separately in future-agi/docs#1093, with released-dependency tests. Fixed-release dependency pins still require genuine package releases. No invented/unpublished version is put in customer installation instructions. This PR alone is not a claim that the released recipe's entire recall is resolved.

Reproduction and verification

All Python tests and builds ran in the prescribed macOS sandbox denying outbound network, with an isolated synthetic HOME and synthetic credentials. SDK requests use fake transports; exact recipe entry-point tests use the real registration, FI processor and loopback OTLP exporter. No vendor or Cloud credentials/calls.

Python / SDK / OTel OpenAI package tests + new core cancellation tests
3.10 / OpenAI 1.69.0 / OTel 1.29.0 449 passed, 2 skipped
3.12 / OpenAI 1.69.0 / OTel 1.29.0 449 passed, 2 skipped
3.12 / OpenAI 3.42.1 / OTel 1.45.2 449 passed, 2 skipped
3.13 / OpenAI 3.42.1 / OTel 1.45.2 449 passed, 2 skipped
Combined with #520, floor SDK 584 passed, 10 skipped
Combined with #520, latest SDK 592 passed, 2 skipped

After installing the optional Fireworks and vLLM compatibility packages, the complete OpenAI suite plus new core cancellation cases passes 643 tests, zero skips on both OpenAI 1.69.0 and 3.42.1 (Python 3.12). The first CI run caught Fireworks' success fixture emitting no terminal choice before its usage tail. Added the missing stop / tool_calls chunk rather than weakening incomplete-stream detection; its 24 failures reproduced locally before the fixture correction. The earlier two module skips in the table are these optional route suites.

Core suite excluding framework adapters and the separate harness: floor 874 passed, 6 skipped, 2 failed; latest 871 passed, 6 skipped, 5 failed (24 subtests passed in each). All failures reproduced on untouched dev: two gRPC loopback exports rejected by the macOS sandbox, plus three latest-OTel tests asserting the removed private exporter _timeout field. The separate harness cannot collect in these environments because tests.harness is shadowed; it is not claimed as covered. No pre-existing failures were hidden by changing tests.

  • Built wheel and sdist distributions for core and OpenAI.
  • pip check passed at both SDK bounds.
  • python3.12 .github/release/release_plan.py --check: 129 packages, versions and coverage consistent.
  • git diff --check passed. No version/lockfile changes; release-please owns the releases.

Self-review found no introduced blocking issue. No Ruff configuration applies to these Python packages; no migrations, application contracts or frontend files changed.

E2E coverage

not applicable (no e2e/ harness) · flows hit: n/a · marker: n/a · verdict: not applicable

Package contracts and actual loopback OTLP export cover the changed SDK behavior. No application/UI or live vendor verification is claimed.

Recall round 2 integration verification

This head includes #520's effective MCP-credential and raw-body-consumption fixes, without losing effective n override validation or core cancellation handling.

  • Combined full OpenAI suite: 803 passed, 8 SDK-feature skips on OpenAI 1.69.0 / OTel 1.29.0; 811 passed on latest OpenAI 3.42.1 / OTel 1.45.2.
  • Core cancellation regressions: 68 passed at each bound.
  • All independent round-2 probes: 14 passed at each bound; TrueFoundry recipe with the application fix: 64 passed at each bound.
  • Existing metadata-contract recipes per bound: OrcaRouter 33, Doubleword 44, Perplexity 187, PrismML 97; all 361 passed without weakening assertions.
  • Wheel/sdist builds and release-plan validation pass. Tests run in the outbound-denying sandbox with isolated HOME and synthetic credentials; no live vendor or Cloud claims.
  • Genuine fixed-release pins remain pending publication; no versions were invented and no registry was published.

Generated with Claude Code

nik13 and others added 12 commits November 1, 2026 21:55
Strip MCP authentication from telemetry-only tool copies, redact known credentials from attributes and exception telemetry, and keep echoed request fields out of response output. Honor failed terminal results, finalize explicit and high-level stream closes, and propagate cancellation after ending spans. Mark interrupted Parallel Task workflows as errors.

Co-Authored-By: Claude Code <noreply@anthropic.com>
Redact resolved request credentials, prevent core traceback reconstruction from restoring secrets, mask nested MCP errors, and trace parsed raw streams. Keep legacy core and published recipe contracts covered.

Co-Authored-By: Claude Code <noreply@anthropic.com>
Co-Authored-By: Claude Code <noreply@anthropic.com>
Capture actual HTTP request credentials in a per-call context before SDK parsing. Preserve lazy async envelope parsing, finalize envelope closes, and trace every explicit stream parse cache entry with a shared terminal result.

Co-Authored-By: Claude Code <noreply@anthropic.com>
…rors

Handle non-Exception aborts through the privacy-aware core tracer. Require terminal finish reasons for every requested chat choice before marking a stream successful. Reuse the stream status hook from PR #520 without duplicating its cancellation and close fixes.

Co-Authored-By: Claude Code <noreply@anthropic.com>
Include a terminal finish reason before the usage tail. Preserve strict truncated-stream detection and run both optional migration-route suites.

Co-Authored-By: Claude Code <noreply@anthropic.com>
Apply extra_body overrides and Omit removals before extracting request metadata and expected streamed choices. Cover sync/async on-wire override directions, removal, null/default counts, and non-mutating shallow merge behavior.

Co-Authored-By: Claude Code <noreply@anthropic.com>
…-1069

Co-Authored-By: Claude Code <noreply@anthropic.com>
Preserve existing exported request metadata and recipe contracts; only the chat stream accumulator consumes the effective body. Verify arbitrary extra_body values are not newly exported while overridden and removed choice counts determine completion status.

Co-Authored-By: Claude Code <noreply@anthropic.com>
Co-Authored-By: Claude Code <noreply@anthropic.com>
Co-Authored-By: Claude Code <noreply@anthropic.com>
Co-Authored-By: Claude Code <noreply@anthropic.com>

# Conflicts:
#	python/frameworks/openai/traceai_openai/_request.py
nik13 added a commit that referenced this pull request Nov 2, 2026
Keep published-package cancellation, early-close and terminal-state contracts strict. Shared instrumentor repairs remain in #520/#536 pending a genuine fixed release.

Co-Authored-By: Claude Code <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant