Skip to content

fix(cohere): trace v2 clients and async chat streams - #346

Open
nik13 wants to merge 7 commits into
devfrom
fix/cohere-v2-async-stream-tracing
Open

nik13 wants to merge 7 commits into
devfrom
fix/cohere-v2-async-stream-tracing

Conversation

@nik13

@nik13 nik13 commented Oct 31, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Keep traceAI-cohere and repair its SDK coverage rather than retire it. ClientV2 and AsyncClientV2 resolve separate SDK implementations, so patching only v1 clients did not trace their calls. Async v1 streaming was also unwrapped.

  • Instrument chat, chat streams, embeddings and reranking on v1/v2 sync/async clients. Resolve and wrap each owning implementation once, including older v2 clients that inherit v1 methods.
  • Preserve CohereInstrumentor().instrument(...), v1 entrypoints, original responses/events, suppression and uninstrumentation.
  • Capture v2 messages, streamed tool arguments, consistent actual-token totals (falling back to billed units), and typed embeddings. Move tool arguments onto canonical message attributes so privacy settings apply.
  • Keep stream context active only during SDK iteration. Finish stream spans on exhaustion, errors/cancellation, explicit close/aclose, or abandonment; preserve original exceptions.
  • Apply the configured core image mask before serializing either input.value or message.content. Respect hide_input_images and base64_image_max_length, preserve text, and never modify the request sent to Cohere. Explicit config takes precedence over environment settings.
  • Honor selective input/output message and text privacy in raw telemetry copies, including v1 response-side chat_history. Native stream error details use the exact effective config before serialization; caller data is unchanged.
  • End cancelled unary requests without changing cancellation, and mark native ERROR, ERROR_TOXIC, TIMEOUT or error-bearing stream endings as failures, including explicit close after a terminal event.
  • Raise the core minimum from the incompatible 0.1.11 to tested 1.2.2. Add installed-distribution CI for SDK/core floors, early v2, and latest releases.
  • Replace v1-only README examples with v2 sync/async examples and explicit content-privacy configuration. No copied customer instrumentation helper, version bump or changelog edit.

Reproduction

The original dev baseline at cdee70dd76ff7ee00aaa69e0d6477485e7bd6925 failed 24 of 32 real-SDK surface/wire regression cases on Cohere 7.3.0. The tests import the baseline implementation explicitly using PYTHONPATH and pytest's --import-mode=importlib; the SDK calls use httpx.MockTransport.

Before the image fix, the new multimodal SDK tests fail 32 of 48 cases: image hiding and base64 limits are bypassed by both serialized attributes. The same cases now pass. They cover sync/async chat and streaming, dictionaries and SDK model objects, and explicit/environment configuration. Assertions inspect both attributes and serialized OTLP bytes, retain text, and prove the provider request remains unchanged.

Independent review reproductions failed 32 of 32 behavioral cases before the review fixes. The old declared core floor passed dependency checks but raised AttributeError before any provider request; the same call succeeds on the new minimum. Expanded tests also reproduced v1 input leakage through the response-side history. The regression suite now covers explicit/environment settings in both directions, async paths, unchanged provider payloads, cancellation identity, terminal failure status, and serialized OTLP non-disclosure.

Merged origin/dev normally into this branch; no force push or duplicate PR.

Tests

All runtime tests ran under an OS outbound-network-deny sandbox, with synthetic credentials and fake HTTP transports. No vendor/model API or Cloud collector was called.

  • Cohere 5.0.0 and core 1.2.2 (declared floors): 154 passed, 177 skipped, on Python 3.10 (installed wheel) and Python 3.12. Skips require unavailable v2/typed-image APIs. The old SDK uses httpx==0.27.2.
  • Cohere 5.13.12 (early v2), Python 3.12: 307 passed, 24 skipped. Skips require typed multimodal models absent from this release.
  • Cohere 7.3.0 and core 1.2.2 (public PyPI latest), Python 3.12 and 3.13: 331 passed. Python 3.13 runs the installed wheel with isolated imports.
  • The compatibility workflow installs distributions and runs the suite in a network namespace without outbound connectivity. Core 1.2.2 is explicitly pinned in floor/early-v2 jobs; the latest job resolves current public releases.
  • OTLP protobuf assertions cover span counts/names, model, messages, tool arguments, usage, embedding dimensions and content/image privacy.
  • Stream lifecycle tests cover parent context, unchanged event identity, original errors, cancellation, early/unstarted/repeated close, abandoned streams, unknown events, suppression, and repeated instrument/uninstrument cycles.
  • All 5 README Python blocks executed with real SDK clients and synthetic HTTP transports.
  • python3 .github/release/release_plan.py --check: 129 packages, versions and coverage consistent.
  • Ruff F checks on changed source/tests, formatter check and git diff --check pass. The pre-existing legacy test file has three unused-import lint warnings, outside the changed files. There is no package-configured Ruff gate.

Re-run the package suite after installing the package, pytest, pytest-asyncio and the OTLP HTTP exporter:

python -m pytest python/frameworks/cohere/tests -q

E2E coverage

Not applicable: this repository has no e2e/scripts/e2e-coverage.mjs app harness. Provider SDK transport-to-OTLP coverage is included in the package suite. Live provider behavior is not claimed.

Release

Targets dev; release-please owns the package version and changelog. No merge to dev, tag, package publication or docs-site PR is part of this change. Release the corrected package before advertising the new coverage separately.

🤖 Generated with Claude Code

nik13 and others added 7 commits October 31, 2026 05:41
Resolve each SDK method owner once, preserve v1 instrumentation, and capture v2 messages, tool deltas, token usage and typed embeddings. Keep streaming span context local to SDK iteration and finish on completion, errors, cancellation or close. Use canonical message attributes for privacy.

Add offline SDK and OTLP-wire regression coverage for Cohere 5.0.0, 5.13.12 and 7.3.0. Update package examples to v2 without changing the public instrumentor API.

Co-Authored-By: Claude Code <noreply@anthropic.com>
Co-Authored-By: Claude Code <noreply@anthropic.com>
Apply the configured core image mask to telemetry-only message copies
so hide_input_images and base64 size limits cover both input.value and
message.content. Preserve original requests and explicit config priority.

Add failing-first real-SDK OTLP regressions for sync/async chat and streams,
typed/dict messages, environment options, and direct floor-SDK extraction.

Co-Authored-By: Claude Code <noreply@anthropic.com>
Honor selective content flags on raw telemetry copies, mask native stream error details with the effective config, finish cancelled calls, and classify terminal provider failures. Require tested core 1.2.2 and exercise SDK/core floors and latest distributions in compatibility CI.

Co-Authored-By: Claude Code <noreply@anthropic.com>
Co-Authored-By: Claude Code <noreply@anthropic.com>
Strip document and tool-result citation payloads only from telemetry copies under effective input privacy. Share native response failure classification across unary and streaming chat, including ERROR_LIMIT. Add SDK-to-OTLP regression coverage for configuration precedence, immutable caller data, and terminal status.

Co-Authored-By: Claude Code <noreply@anthropic.com>
Co-Authored-By: Claude Code <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant