Skip to content

fix(deepseek): migrate to the shared OpenAI tracing route - #376

Open
nik13 wants to merge 8 commits into
devfrom
fix/deepseek-python-shared-route
Open

nik13 wants to merge 8 commits into
devfrom
fix/deepseek-python-shared-route

Conversation

@nik13

@nik13 nik13 commented Oct 31, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Use the shared OpenAI integration for DeepSeek, without deleting the existing package or dropping its reasoning/cache fields.

  • Add host-scoped DeepSeek extraction to traceAI-openai: reasoning text and token counts, prompt-cache hit/miss counts, response ID, finish reason, and tool-call aliases. Output aliases and standard tool fields obey the same output privacy controls as the legacy integration.

  • Accumulate fragmented reasoning and tool arguments using the existing indexed/string accumulator primitives. Only DeepSeek requests select this accumulator; no second set of raw stream chunks is retained.

  • Keep DeepSeekInstrumentor available, with a deprecation warning, for existing Chat Completions applications and OpenAI SDK 1.0–1.68. Replace its generators with stream proxies supporting SDK context managers, explicit close, async close, early exits, and original response access. Match the exact API host rather than URL substrings.

  • When both updated integrations are active, the shared integration owns the request, including its provider/privacy settings. Both initialization orders and independent teardown work. A capability marker prevents the new legacy package from silently handing off to an old shared release that lacks DeepSeek extraction; that old combination retains its existing duplicate-span behavior until upgraded.

  • Recommend the shared route for both Chat Completions and Responses. The legacy entry point remains Chat Completions-only, explicitly documented rather than pretending it supports Responses.

  • Finish shared streamed spans on explicit close() as well as exhaustion/context exit and cancellation. Cancellation is re-raised unchanged; later close cannot mark it successful or export a second span.

  • Apply output-text privacy to DeepSeek Responses function-call arguments. Retain completed, incomplete, and failed terminal responses with their output, usage, response ID, and reasoning counts. Failed streamed and non-streamed Responses are errors, and vendor error details are excluded from raw output. No SDK response or exception is replaced by telemetry data.

  • Finish cancellation while awaiting create() before headers arrive, for both async Chat Completions and Responses; preserve the original cancellation and error-detail masking.

  • Extract Responses reasoning.content separately from the final assistant answer, handling dictionary extras on older SDKs and typed items on newer SDKs. Keep summaries alongside content and apply the existing output privacy policy. Streamed answer previews now include only assistant output_text items.

Why this route

DeepSeek documents the OpenAI-compatible Responses API, using the same https://api.deepseek.com client base URL. Braintrust's shared OpenAI wrapper checks existing wrapping to avoid duplicate spans. Both sources were fetched independently during this repair; no vendor implementation was copied. This change retains our existing process-wide OpenAI instrumentor API rather than introducing an instance wrapper API.

The independent review's three findings were confirmed against the reviewed PR head a14984ae2331b944b698f2f3f0601217aac144e3. The final regression selection produced 30 failed / 30 passed at OpenAI 1.69.0, OpenAI 3.42.1, and OpenAI 3.42.1 with released core 1.2.2. These are parameterized cases, not 30 distinct defects. After the fixes, all 60 new cases pass in each environment.

The Responses privacy and terminal-event defects were inherited shared-code gaps exposed by the newly recommended route. The cancellation defect became a migration regression when the legacy instrumentor handed ownership to the shared stream.

Round-two review also confirmed two inherited residual gaps: cancellation before create() returns and plaintext Responses reasoning. Against reviewed head 1836aa69, the new regressions produced 44 failures at latest SDK/released core, 36 at the shared floor, and 4 at the legacy floor. All applicable new cases pass after the repair: 60 at each shared SDK/core boundary, plus 4 legacy-floor cancellation cases. The original independent round-two probes also pass all 28 cases at each shared boundary. These counts are parameterized cases, not distinct defects.

Verification

All local test executions ran under a macOS network sandbox denying outbound networking except localhost. Model HTTP uses httpx.MockTransport with the real SDK. The migration suite decodes actual OTLP protobuf received over a localhost HTTP server.

Final local results for the repaired Python sources, after normal merges of dev through b059859f:

Environment Shared package suite, including migration contracts Existing DeepSeek suite
OpenAI 3.42.1 / Python 3.12 / source core 570 passed, 6 skipped 71 passed
OpenAI 1.69.0 / Python 3.10 / source core 570 passed, 6 skipped 71 passed
OpenAI 3.42.1 / Python 3.12 / released core 1.2.2 570 passed, 6 skipped 71 passed
OpenAI 1.0.0 / Python 3.10 / source core, no shared package 34 legacy migration cases passed, 215 inapplicable cases skipped 71 passed

OpenAI 3.42.1 is the latest stable release reported by public PyPI metadata. The release-plan check passes for 129 packages, and the complete PR diff passes git diff --check. No versions, release manifests, or CHANGELOG release sections were manually bumped.

The migration contract covers synchronous/asynchronous Chat Completions, streams, Responses, reasoning/cache/tool fields, output privacy on the wire, API-error privacy with original error propagation, early/empty/explicit stream close, cancellation before headers and during iteration, separate Responses reasoning content/summary and final answer, exact-host rejection, repeated instrumentation, initialization/teardown ordering, shared configuration ownership, old-shared capability negotiation, non-DeepSeek isolation, and explicit privacy configuration overriding the environment.

Earlier migration validation ran the README and held MDX examples with the real register(), mocked vendor HTTP, and localhost OTLP on both shared SDK boundaries and released core 1.2.2. Each emitted three correctly attributed, content-masked spans. The held MDX compiles. No live model API or Future AGI backend was called, so live backend rendering and a product screenshot are not claimed.

The complete PR diff passes git diff --check. The source package pyprojects do not configure Ruff; there are no migrations, frontend contracts, or OSS/EE-boundary changes. Self-review corrected the old-shared handoff regression before opening this PR.

Rerun the contract with the source core, source DeepSeek/OpenAI packages, pytest, pytest-asyncio, and httpx installed:

python -I -m pytest -q python/frameworks/openai/tests
python -I -m pytest -q python/frameworks/deepseek/tests

For legacy OpenAI 1.0, omit the shared package and run python -I -m pytest -q --import-mode=importlib python/frameworks/openai/tests/test_deepseek_route.py plus the DeepSeek suite separately. The new workflow installs each SDK boundary explicitly and runs all tests in a network namespace with only loopback enabled.

Release and migration

Release the corrected shared package before publishing the deprecated legacy-package update or the migration page. Keep both packages/release inventory: immediate deletion is not safe for existing imports and the older SDK floor. Versions and CHANGELOG release sections are untouched; release-please owns them.

Migration changes are explicit: shared span names are ChatCompletion / Response, output.value is a text/tool summary instead of raw legacy JSON, and request tools use shared semantic attributes instead of deepseek.tools_count. Update saved filters before uninstalling the legacy package.

The customer docs page is held until the corrected package is on PyPI; no docs PR advertises unreleased behavior. TypeScript DeepSeek and its in-flight build/resource fixes are untouched. DeepSeek-specific field aliases remain host-scoped. Responses terminal status handling, cancellation finalization, and raw vendor-error exclusion are corrected in the shared package.

E2E coverage

not applicable (no e2e/ harness) · flows hit: n/a · marker: n/a · verdict: not applicable

Generated with Claude Code

nik13 and others added 8 commits October 31, 2026 07:04
Retain the deprecated legacy entry point and SDK floor, preserve stream lifetime, and give the parity-capable shared instrumentor single-span ownership. Add network-isolated real SDK and OTLP contracts.

Co-Authored-By: Claude Code <noreply@anthropic.com>
Co-Authored-By: Claude Code <noreply@anthropic.com>
Run the shared-route contracts on OpenAI 3.42.1 with source and released core, retaining both SDK floors.

Co-Authored-By: Claude Code <noreply@anthropic.com>
Mask DeepSeek Responses tool output under the configured text policy, retain incomplete and failed terminal payloads, and keep cancellation finalization idempotent. Exclude vendor errors from raw output and report failed Responses as errors without changing SDK return values. Add real-SDK localhost OTLP contracts across sync, async, stream, and non-stream routes.

Co-Authored-By: Claude Code <noreply@anthropic.com>
Co-Authored-By: Claude Code <noreply@anthropic.com>
…oning

Finish async request spans when cancellation occurs before create returns, retaining exception identity and configured error privacy. Extract typed and dictionary reasoning content separately from assistant output and retain reasoning summaries.

Co-Authored-By: Claude Code <noreply@anthropic.com>
Co-Authored-By: Claude Code <noreply@anthropic.com>
Co-Authored-By: Claude Code <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant